[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790205259943518-25487-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 23:14:19 dist-test-slave-754n krb5kdc[25494](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 23:14:19 dist-test-slave-754n krb5kdc[25494](info): set up 2 sockets
Sep 23 23:14:19 dist-test-slave-754n krb5kdc[25494](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260923 23:14:20.962509 25487 mini_kdc.cc:121] Time spent starting KDC: real 1.002s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790205259943518-25487-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790205259943518-25487-0/krb5kdc/test-user.keytab.
Sep 23 23:14:21 dist-test-slave-754n krb5kdc[25494](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205261, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T23:14:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T23:14:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.227.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.227.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.227.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab.
Entry for principal kudu/127.24.227.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab.
I20260923 23:14:21.074942 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790205259943518-25487-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790205259943518-25487-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790205259943518-25487-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.254:42135
--webserver_interface=127.24.227.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab
--principal=kudu/127.24.227.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:39531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.227.254:42135
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:14:21.152680 25507 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:14:21.152899 25507 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:14:21.152935 25507 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:14:21.154021 25507 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 23:14:21.154084 25507 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:14:21.154109 25507 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:14:21.154129 25507 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 23:14:21.154156 25507 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 23:14:21.155498 25507 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:39531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790205259943518-25487-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790205259943518-25487-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.227.254:42135
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab
--principal=kudu/127.24.227.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.254:42135
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.227.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25507
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790205259943518-25487-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:14:21.155742 25507 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:14:21.155961 25507 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:14:21.160717 25515 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:21.160782 25513 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:21.160782 25512 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:14:21.160979 25507 server_base.cc:1034] running on GCE node
I20260923 23:14:21.161181 25507 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:14:21.161439 25507 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:14:21.162573 25507 hybrid_clock.cc:648] HybridClock initialized: now 1790205261162556 us; error 33 us; skew 500 ppm
Sep 23 23:14:21 dist-test-slave-754n krb5kdc[25494](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205261, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:14:21.164958 25507 init.cc:377] Logged in from keytab as kudu/127.24.227.254@KRBTEST.COM (short username kudu)
I20260923 23:14:21.165287 25507 webserver.cc:460] Webserver started at http://127.24.227.254:43369/ using document root <none> and password file <none>
I20260923 23:14:21.165464 25507 fs_manager.cc:359] Metadata directory not provided
I20260923 23:14:21.165503 25507 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:14:21.165585 25507 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:14:21.166476 25507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790205259943518-25487-0/minicluster-data/master-0/data/instance:
uuid: "9526893056e34caaab693e079ac05fde"
format_stamp: "Formatted at 2026-09-23 23:14:21 on dist-test-slave-754n"
server_key: "ae076a3f05fe70c564c1fb69148eafeb"
server_key_iv: "a6f3ab843e46f2b93a2b3d8970a17cdd"
server_key_version: "encryptionkey@0"
I20260923 23:14:21.166754 25507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790205259943518-25487-0/minicluster-data/master-0/wal/instance:
uuid: "9526893056e34caaab693e079ac05fde"
format_stamp: "Formatted at 2026-09-23 23:14:21 on dist-test-slave-754n"
server_key: "ae076a3f05fe70c564c1fb69148eafeb"
server_key_iv: "a6f3ab843e46f2b93a2b3d8970a17cdd"
server_key_version: "encryptionkey@0"
I20260923 23:14:21.168084 25507 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 23:14:21.168766 25522 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:21.168928 25507 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 23:14:21.169024 25507 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790205259943518-25487-0/minicluster-data/master-0/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790205259943518-25487-0/minicluster-data/master-0/wal
uuid: "9526893056e34caaab693e079ac05fde"
format_stamp: "Formatted at 2026-09-23 23:14:21 on dist-test-slave-754n"
server_key: "ae076a3f05fe70c564c1fb69148eafeb"
server_key_iv: "a6f3ab843e46f2b93a2b3d8970a17cdd"
server_key_version: "encryptionkey@0"
I20260923 23:14:21.169100 25507 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790205259943518-25487-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790205259943518-25487-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790205259943518-25487-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:14:21.181813 25507 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:14:21.183498 25507 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:14:21.183642 25507 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:14:21.187369 25507 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.254:42135
I20260923 23:14:21.187400 25574 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.254:42135 every 8 connection(s)
I20260923 23:14:21.187726 25507 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
I20260923 23:14:21.188655 25575 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:14:21.188973 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 25507
I20260923 23:14:21.189131 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790205259943518-25487-0/minicluster-data/master-0/wal/instance
I20260923 23:14:21.189764 25487 external_mini_cluster.cc:1428] Setting key 842d40152fd45aef4eebd1433ea485c1
I20260923 23:14:21.191242 25575 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9526893056e34caaab693e079ac05fde: Bootstrap starting.
I20260923 23:14:21.191938 25575 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9526893056e34caaab693e079ac05fde: Neither blocks nor log segments found. Creating new log.
I20260923 23:14:21.192245 25575 log.cc:824] T 00000000000000000000000000000000 P 9526893056e34caaab693e079ac05fde: Log is configured to *not* fsync() on all Append() calls
I20260923 23:14:21.192981 25575 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9526893056e34caaab693e079ac05fde: No bootstrap required, opened a new log
I20260923 23:14:21.194983 25575 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9526893056e34caaab693e079ac05fde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9526893056e34caaab693e079ac05fde" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 42135 } }
I20260923 23:14:21.195142 25575 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9526893056e34caaab693e079ac05fde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:14:21.195176 25575 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9526893056e34caaab693e079ac05fde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9526893056e34caaab693e079ac05fde, State: Initialized, Role: FOLLOWER
I20260923 23:14:21.195298 25575 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9526893056e34caaab693e079ac05fde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9526893056e34caaab693e079ac05fde" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 42135 } }
I20260923 23:14:21.195374 25575 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9526893056e34caaab693e079ac05fde [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 23:14:21.195406 25575 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9526893056e34caaab693e079ac05fde [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 23:14:21.195456 25575 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9526893056e34caaab693e079ac05fde [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:14:21.196506 25575 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9526893056e34caaab693e079ac05fde [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9526893056e34caaab693e079ac05fde" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 42135 } }
I20260923 23:14:21.196593 25575 leader_election.cc:302] T 00000000000000000000000000000000 P 9526893056e34caaab693e079ac05fde [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9526893056e34caaab693e079ac05fde; no voters:
I20260923 23:14:21.196811 25575 leader_election.cc:288] T 00000000000000000000000000000000 P 9526893056e34caaab693e079ac05fde [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 23:14:21.196834 25580 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9526893056e34caaab693e079ac05fde [term 1 FOLLOWER]: Leader election won for term 1
I20260923 23:14:21.196941 25580 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9526893056e34caaab693e079ac05fde [term 1 LEADER]: Becoming Leader. State: Replica: 9526893056e34caaab693e079ac05fde, State: Running, Role: LEADER
I20260923 23:14:21.197037 25580 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9526893056e34caaab693e079ac05fde [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9526893056e34caaab693e079ac05fde" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 42135 } }
I20260923 23:14:21.197080 25575 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9526893056e34caaab693e079ac05fde [sys.catalog]: configured and running, proceeding with master startup.
I20260923 23:14:21.197369 25580 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9526893056e34caaab693e079ac05fde [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9526893056e34caaab693e079ac05fde" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9526893056e34caaab693e079ac05fde" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 42135 } } }
I20260923 23:14:21.197449 25580 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9526893056e34caaab693e079ac05fde [sys.catalog]: This master's current role is: LEADER
I20260923 23:14:21.197568 25580 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9526893056e34caaab693e079ac05fde [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9526893056e34caaab693e079ac05fde. Latest consensus state: current_term: 1 leader_uuid: "9526893056e34caaab693e079ac05fde" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9526893056e34caaab693e079ac05fde" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 42135 } } }
I20260923 23:14:21.197636 25580 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9526893056e34caaab693e079ac05fde [sys.catalog]: This master's current role is: LEADER
I20260923 23:14:21.197767 25585 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 23:14:21.198487 25585 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 23:14:21.200519 25585 catalog_manager.cc:1347] Generated new cluster ID: 7e9c57ae73d94962947aeaf31c6da118
I20260923 23:14:21.200585 25585 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 23:14:21 dist-test-slave-754n krb5kdc[25494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205261, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
I20260923 23:14:21.207765 25578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:21.190535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32820 (local address 127.24.227.254:42135)
0923 23:14:21.190712 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:21.190721 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:21.190914 (+ 193us) server_negotiation.cc:407] Connection header received
0923 23:14:21.194065 (+ 3151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:21.194071 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:21.194114 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:21.194217 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:21.196333 (+ 2116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:21.197612 (+ 1279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:21.199048 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:21.199616 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:21.199734 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:21.203729 (+ 3995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:21.203733 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:21.203733 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:21.203744 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:21.205849 (+ 2105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:21.206303 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:21.206306 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:21.206306 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:21.206419 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:21.206701 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:21.206702 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:21.206703 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:21.206991 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:21.207142 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:21.207433 (+ 291us) server_negotiation.cc:299] Negotiation successful
0923 23:14:21.207566 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":65,"threads_started":1}
I20260923 23:14:21.209115 25585 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 23:14:21.209575 25585 catalog_manager.cc:1504] Loading token signing keys...
I20260923 23:14:21.218636 25585 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9526893056e34caaab693e079ac05fde: Generated new TSK 0
I20260923 23:14:21.218807 25585 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.227.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.227.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.227.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.193.keytab.
Entry for principal kudu/127.24.227.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.193.keytab.
I20260923 23:14:21.250098 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790205259943518-25487-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790205259943518-25487-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790205259943518-25487-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790205259943518-25487-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.227.193:0
--local_ip_for_outbound_sockets=127.24.227.193
--webserver_interface=127.24.227.193
--webserver_port=0
--tserver_master_addrs=127.24.227.254:42135
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.193.keytab
--principal=kudu/127.24.227.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:39531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:14:21.329289 25601 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:14:21.329516 25601 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:14:21.329540 25601 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:14:21.330576 25601 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:14:21.330610 25601 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:14:21.330634 25601 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.227.193
I20260923 23:14:21.331977 25601 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:39531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790205259943518-25487-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790205259943518-25487-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.193.keytab
--principal=kudu/127.24.227.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.227.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790205259943518-25487-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.227.193
--webserver_port=0
--tserver_master_addrs=127.24.227.254:42135
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25601
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.227.193
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790205259943518-25487-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:14:21.332146 25601 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:14:21.332307 25601 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:14:21.336926 25609 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:21.336938 25607 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:14:21.337291 25601 server_base.cc:1034] running on GCE node
W20260923 23:14:21.336956 25606 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:14:21.337600 25601 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:14:21.337853 25601 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:14:21.339025 25601 hybrid_clock.cc:648] HybridClock initialized: now 1790205261339023 us; error 40 us; skew 500 ppm
Sep 23 23:14:21 dist-test-slave-754n krb5kdc[25494](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205261, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:14:21.341544 25601 init.cc:377] Logged in from keytab as kudu/127.24.227.193@KRBTEST.COM (short username kudu)
I20260923 23:14:21.341969 25601 webserver.cc:460] Webserver started at http://127.24.227.193:44389/ using document root <none> and password file <none>
I20260923 23:14:21.342204 25601 fs_manager.cc:359] Metadata directory not provided
I20260923 23:14:21.342274 25601 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:14:21.342429 25601 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:14:21.343487 25601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790205259943518-25487-0/minicluster-data/ts-0/data/instance:
uuid: "ff611302d3ef4683a83d2b68254d21b1"
format_stamp: "Formatted at 2026-09-23 23:14:21 on dist-test-slave-754n"
server_key: "ff9ba9861ac40d2bc63374638094ae36"
server_key_iv: "b76f12c543773a33b2e0ccae1e63090c"
server_key_version: "encryptionkey@0"
I20260923 23:14:21.343843 25601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790205259943518-25487-0/minicluster-data/ts-0/wal/instance:
uuid: "ff611302d3ef4683a83d2b68254d21b1"
format_stamp: "Formatted at 2026-09-23 23:14:21 on dist-test-slave-754n"
server_key: "ff9ba9861ac40d2bc63374638094ae36"
server_key_iv: "b76f12c543773a33b2e0ccae1e63090c"
server_key_version: "encryptionkey@0"
I20260923 23:14:21.345302 25601 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 23:14:21.346025 25616 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:21.346196 25601 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 23:14:21.346292 25601 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790205259943518-25487-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790205259943518-25487-0/minicluster-data/ts-0/wal
uuid: "ff611302d3ef4683a83d2b68254d21b1"
format_stamp: "Formatted at 2026-09-23 23:14:21 on dist-test-slave-754n"
server_key: "ff9ba9861ac40d2bc63374638094ae36"
server_key_iv: "b76f12c543773a33b2e0ccae1e63090c"
server_key_version: "encryptionkey@0"
I20260923 23:14:21.346369 25601 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790205259943518-25487-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790205259943518-25487-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790205259943518-25487-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:14:21.357123 25601 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:14:21.358783 25601 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:14:21.358933 25601 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:14:21.359135 25601 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 23:14:21.359439 25601 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 23:14:21.359493 25601 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:21.359530 25601 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 23:14:21.359556 25601 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:21.365586 25601 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.193:39885
I20260923 23:14:21.365631 25729 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.193:39885 every 8 connection(s)
I20260923 23:14:21.365895 25601 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790205259943518-25487-0/minicluster-data/ts-0/data/info.pb
Sep 23 23:14:21 dist-test-slave-754n krb5kdc[25494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205261, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.193@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
I20260923 23:14:21.374152 25578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:21.366553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.193:40111 (local address 127.24.227.254:42135)
0923 23:14:21.366610 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:21.366612 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:21.366787 (+ 175us) server_negotiation.cc:407] Connection header received
0923 23:14:21.367235 (+ 448us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:21.367236 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:21.367244 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:21.367298 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:21.368240 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:21.368713 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:21.369563 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:21.369763 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:21.369814 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:21.371716 (+ 1902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:21.371717 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:21.371717 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:21.371723 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:21.373121 (+ 1398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:21.373463 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:21.373464 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:21.373464 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:21.373510 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:21.373700 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:21.373701 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:21.373701 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:21.373798 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:21.373843 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:21.374040 (+ 197us) server_negotiation.cc:299] Negotiation successful
0923 23:14:21.374095 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260923 23:14:21.374300 25732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:21.366509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:42135 (local address 127.24.227.193:40111)
0923 23:14:21.366650 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:21.366688 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:21.367158 (+ 470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:21.367344 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:21.367347 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:21.367968 (+ 621us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:14:21.368148 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:21.368153 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:21.368795 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:21.368795 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:21.369498 (+ 703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:21.369499 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:21.369829 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:21.369830 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:21.369920 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:21.370322 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:14:21.370329 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:14:21.371614 (+ 1285us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:14:21.373186 (+ 1572us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:21.373191 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:21.373193 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:21.373394 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:21.373564 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:21.373565 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:21.373565 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:21.373650 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:21.373846 (+ 196us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:14:21.373849 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:14:21.373946 (+ 97us) client_negotiation.cc:769] Sending connection context
0923 23:14:21.374017 (+ 71us) client_negotiation.cc:240] Negotiation successful
0923 23:14:21.374113 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":78,"threads_started":1}
I20260923 23:14:21.374454 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 25601
I20260923 23:14:21.374568 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790205259943518-25487-0/minicluster-data/ts-0/wal/instance
I20260923 23:14:21.374743 25487 external_mini_cluster.cc:1428] Setting key d5b183ac30ee2701ec195e49aabe841c
I20260923 23:14:21.374819 25730 heartbeater.cc:347] Connected to a master server at 127.24.227.254:42135
I20260923 23:14:21.374922 25730 heartbeater.cc:464] Registering TS with master...
I20260923 23:14:21.375100 25730 heartbeater.cc:511] Master 127.24.227.254:42135 requested a full tablet report, sending...
I20260923 23:14:21.375581 25539 ts_manager.cc:194] Registered new tserver with Master: ff611302d3ef4683a83d2b68254d21b1 (127.24.227.193:39885)
I20260923 23:14:21.376224 25539 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.227.193@KRBTEST.COM'} at 127.24.227.193:40111
WARNING: no policy specified for kudu/127.24.227.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.227.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.227.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.194.keytab.
Entry for principal kudu/127.24.227.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.194.keytab.
I20260923 23:14:21.398540 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790205259943518-25487-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790205259943518-25487-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790205259943518-25487-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790205259943518-25487-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.227.194:0
--local_ip_for_outbound_sockets=127.24.227.194
--webserver_interface=127.24.227.194
--webserver_port=0
--tserver_master_addrs=127.24.227.254:42135
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.194.keytab
--principal=kudu/127.24.227.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:39531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:14:21.478266 25735 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:14:21.478569 25735 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:14:21.478633 25735 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:14:21.480659 25735 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:14:21.480731 25735 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:14:21.480777 25735 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.227.194
I20260923 23:14:21.483094 25735 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:39531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790205259943518-25487-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790205259943518-25487-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.194.keytab
--principal=kudu/127.24.227.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.227.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790205259943518-25487-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.227.194
--webserver_port=0
--tserver_master_addrs=127.24.227.254:42135
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25735
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.227.194
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790205259943518-25487-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:14:21.483325 25735 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:14:21.483582 25735 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:14:21.488711 25743 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:21.488756 25740 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:21.488729 25741 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:14:21.489048 25735 server_base.cc:1034] running on GCE node
I20260923 23:14:21.489244 25735 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:14:21.489499 25735 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:14:21.490657 25735 hybrid_clock.cc:648] HybridClock initialized: now 1790205261490648 us; error 40 us; skew 500 ppm
Sep 23 23:14:21 dist-test-slave-754n krb5kdc[25494](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205261, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:14:21.493111 25735 init.cc:377] Logged in from keytab as kudu/127.24.227.194@KRBTEST.COM (short username kudu)
I20260923 23:14:21.493474 25735 webserver.cc:460] Webserver started at http://127.24.227.194:39059/ using document root <none> and password file <none>
I20260923 23:14:21.493662 25735 fs_manager.cc:359] Metadata directory not provided
I20260923 23:14:21.493701 25735 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:14:21.493782 25735 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:14:21.494721 25735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790205259943518-25487-0/minicluster-data/ts-1/data/instance:
uuid: "2c336ee66e514311b34c17d0c6044eed"
format_stamp: "Formatted at 2026-09-23 23:14:21 on dist-test-slave-754n"
server_key: "23ed6c4b6c93f8d53905f602725821b2"
server_key_iv: "36ee654d987500eb54e19c2bde42d76b"
server_key_version: "encryptionkey@0"
I20260923 23:14:21.495010 25735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790205259943518-25487-0/minicluster-data/ts-1/wal/instance:
uuid: "2c336ee66e514311b34c17d0c6044eed"
format_stamp: "Formatted at 2026-09-23 23:14:21 on dist-test-slave-754n"
server_key: "23ed6c4b6c93f8d53905f602725821b2"
server_key_iv: "36ee654d987500eb54e19c2bde42d76b"
server_key_version: "encryptionkey@0"
I20260923 23:14:21.496325 25735 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 23:14:21.496940 25750 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:21.497172 25735 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 23:14:21.497277 25735 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790205259943518-25487-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790205259943518-25487-0/minicluster-data/ts-1/wal
uuid: "2c336ee66e514311b34c17d0c6044eed"
format_stamp: "Formatted at 2026-09-23 23:14:21 on dist-test-slave-754n"
server_key: "23ed6c4b6c93f8d53905f602725821b2"
server_key_iv: "36ee654d987500eb54e19c2bde42d76b"
server_key_version: "encryptionkey@0"
I20260923 23:14:21.497350 25735 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790205259943518-25487-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790205259943518-25487-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790205259943518-25487-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:14:21.513588 25735 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:14:21.515797 25735 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:14:21.515945 25735 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:14:21.516166 25735 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 23:14:21.516459 25735 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 23:14:21.516527 25735 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:21.516568 25735 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 23:14:21.516592 25735 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:21.522259 25735 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.194:33981
I20260923 23:14:21.522310 25863 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.194:33981 every 8 connection(s)
I20260923 23:14:21.522605 25735 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790205259943518-25487-0/minicluster-data/ts-1/data/info.pb
I20260923 23:14:21.522948 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 25735
I20260923 23:14:21.523051 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790205259943518-25487-0/minicluster-data/ts-1/wal/instance
I20260923 23:14:21.523238 25487 external_mini_cluster.cc:1428] Setting key 09c7466146b9d2ff132fdc2858720b98
Sep 23 23:14:21 dist-test-slave-754n krb5kdc[25494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205261, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.194@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.24.227.195@KRBTEST.COM; defaulting to no policy
I20260923 23:14:21.532016 25578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:21.523423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.194:51479 (local address 127.24.227.254:42135)
0923 23:14:21.523470 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:21.523472 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:21.523595 (+ 123us) server_negotiation.cc:407] Connection header received
0923 23:14:21.524199 (+ 604us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:21.524200 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:21.524213 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:21.524260 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:21.525170 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:21.525664 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:21.526628 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:21.526837 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:21.526920 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:21.529347 (+ 2427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:21.529349 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:21.529349 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:21.529355 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:21.530678 (+ 1323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:21.531147 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:21.531148 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:21.531149 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:21.531211 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:21.531477 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:21.531478 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:21.531478 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:21.531578 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:21.531627 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:21.531850 (+ 223us) server_negotiation.cc:299] Negotiation successful
0923 23:14:21.531953 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260923 23:14:21.532192 25866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:21.523264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:42135 (local address 127.24.227.194:51479)
0923 23:14:21.523451 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:21.523495 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:21.524101 (+ 606us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:21.524334 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:21.524338 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:21.524869 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:14:21.525068 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:21.525072 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:21.525754 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:21.525755 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:21.526529 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:21.526531 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:21.526929 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:21.526930 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:21.527064 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:21.527554 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:14:21.527562 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:14:21.529257 (+ 1695us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:14:21.530760 (+ 1503us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:21.530766 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:21.530768 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:21.531054 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:21.531274 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:21.531275 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:21.531275 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:21.531413 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:21.531635 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:14:21.531639 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:14:21.531746 (+ 107us) client_negotiation.cc:769] Sending connection context
0923 23:14:21.531824 (+ 78us) client_negotiation.cc:240] Negotiation successful
0923 23:14:21.532005 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":90,"threads_started":1}
I20260923 23:14:21.532605 25864 heartbeater.cc:347] Connected to a master server at 127.24.227.254:42135
I20260923 23:14:21.532717 25864 heartbeater.cc:464] Registering TS with master...
I20260923 23:14:21.532845 25864 heartbeater.cc:511] Master 127.24.227.254:42135 requested a full tablet report, sending...
I20260923 23:14:21.533172 25539 ts_manager.cc:194] Registered new tserver with Master: 2c336ee66e514311b34c17d0c6044eed (127.24.227.194:33981)
I20260923 23:14:21.533484 25539 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.227.194@KRBTEST.COM'} at 127.24.227.194:51479
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.227.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.227.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.195.keytab.
Entry for principal kudu/127.24.227.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.195.keytab.
I20260923 23:14:21.546761 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790205259943518-25487-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790205259943518-25487-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790205259943518-25487-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790205259943518-25487-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.227.195:0
--local_ip_for_outbound_sockets=127.24.227.195
--webserver_interface=127.24.227.195
--webserver_port=0
--tserver_master_addrs=127.24.227.254:42135
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.195.keytab
--principal=kudu/127.24.227.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:39531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:14:21.624555 25869 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:14:21.624773 25869 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:14:21.624809 25869 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:14:21.625950 25869 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:14:21.626034 25869 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:14:21.626080 25869 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.227.195
I20260923 23:14:21.627517 25869 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:39531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790205259943518-25487-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790205259943518-25487-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.195.keytab
--principal=kudu/127.24.227.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.227.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790205259943518-25487-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.227.195
--webserver_port=0
--tserver_master_addrs=127.24.227.254:42135
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25869
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.227.195
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790205259943518-25487-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:14:21.627758 25869 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:14:21.628000 25869 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260923 23:14:21.632726 25869 server_base.cc:1034] running on GCE node
W20260923 23:14:21.632697 25877 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:21.632684 25875 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:21.632675 25874 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:14:21.633064 25869 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:14:21.633299 25869 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:14:21.634457 25869 hybrid_clock.cc:648] HybridClock initialized: now 1790205261634451 us; error 34 us; skew 500 ppm
Sep 23 23:14:21 dist-test-slave-754n krb5kdc[25494](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205261, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:14:21.636916 25869 init.cc:377] Logged in from keytab as kudu/127.24.227.195@KRBTEST.COM (short username kudu)
I20260923 23:14:21.637322 25869 webserver.cc:460] Webserver started at http://127.24.227.195:32919/ using document root <none> and password file <none>
I20260923 23:14:21.637549 25869 fs_manager.cc:359] Metadata directory not provided
I20260923 23:14:21.637616 25869 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:14:21.637785 25869 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:14:21.638772 25869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790205259943518-25487-0/minicluster-data/ts-2/data/instance:
uuid: "2917076af3c14aa6b51d4b1e706443a8"
format_stamp: "Formatted at 2026-09-23 23:14:21 on dist-test-slave-754n"
server_key: "7c273257b1c95e9e4874b35e07de9bcc"
server_key_iv: "c7e7abd1bd0b98575b720658b665c2d3"
server_key_version: "encryptionkey@0"
I20260923 23:14:21.639076 25869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790205259943518-25487-0/minicluster-data/ts-2/wal/instance:
uuid: "2917076af3c14aa6b51d4b1e706443a8"
format_stamp: "Formatted at 2026-09-23 23:14:21 on dist-test-slave-754n"
server_key: "7c273257b1c95e9e4874b35e07de9bcc"
server_key_iv: "c7e7abd1bd0b98575b720658b665c2d3"
server_key_version: "encryptionkey@0"
I20260923 23:14:21.640378 25869 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 23:14:21.641060 25884 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:21.641225 25869 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:21.641319 25869 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790205259943518-25487-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790205259943518-25487-0/minicluster-data/ts-2/wal
uuid: "2917076af3c14aa6b51d4b1e706443a8"
format_stamp: "Formatted at 2026-09-23 23:14:21 on dist-test-slave-754n"
server_key: "7c273257b1c95e9e4874b35e07de9bcc"
server_key_iv: "c7e7abd1bd0b98575b720658b665c2d3"
server_key_version: "encryptionkey@0"
I20260923 23:14:21.641399 25869 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790205259943518-25487-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790205259943518-25487-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790205259943518-25487-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:14:21.660379 25869 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:14:21.662246 25869 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:14:21.662406 25869 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:14:21.662637 25869 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 23:14:21.662916 25869 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 23:14:21.662966 25869 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:21.662998 25869 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 23:14:21.663028 25869 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:21.668613 25869 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.195:43191
I20260923 23:14:21.668655 25997 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.195:43191 every 8 connection(s)
I20260923 23:14:21.669028 25869 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790205259943518-25487-0/minicluster-data/ts-2/data/info.pb
I20260923 23:14:21.671027 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 25869
I20260923 23:14:21.671149 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790205259943518-25487-0/minicluster-data/ts-2/wal/instance
I20260923 23:14:21.671375 25487 external_mini_cluster.cc:1428] Setting key 560d187d9be374b4625e99742df4b1e6
Sep 23 23:14:21 dist-test-slave-754n krb5kdc[25494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205261, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.195@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
I20260923 23:14:21.677821 25578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:21.669675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.195:42907 (local address 127.24.227.254:42135)
0923 23:14:21.669729 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:21.669731 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:21.669928 (+ 197us) server_negotiation.cc:407] Connection header received
0923 23:14:21.670445 (+ 517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:21.670447 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:21.670458 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:21.670514 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:21.671629 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:21.672294 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:21.673269 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:21.673449 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:21.673508 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:21.675492 (+ 1984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:21.675493 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:21.675493 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:21.675498 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:21.676756 (+ 1258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:21.677089 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:21.677089 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:21.677090 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:21.677135 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:21.677347 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:21.677347 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:21.677348 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:21.677436 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:21.677494 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:21.677705 (+ 211us) server_negotiation.cc:299] Negotiation successful
0923 23:14:21.677761 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260923 23:14:21.677973 26000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:21.669642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:42135 (local address 127.24.227.195:42907)
0923 23:14:21.669819 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:21.669852 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:21.670345 (+ 493us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:21.670610 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:21.670613 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:21.671309 (+ 696us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:14:21.671541 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:21.671545 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:21.672379 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:21.672380 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:21.673194 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:21.673196 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:21.673530 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:21.673531 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:21.673625 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:21.674135 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:14:21.674143 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:14:21.675416 (+ 1273us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:14:21.676823 (+ 1407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:21.676828 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:21.676829 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:21.677025 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:21.677189 (+ 164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:21.677190 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:21.677190 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:21.677296 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:21.677497 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:14:21.677500 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:14:21.677605 (+ 105us) client_negotiation.cc:769] Sending connection context
0923 23:14:21.677682 (+ 77us) client_negotiation.cc:240] Negotiation successful
0923 23:14:21.677789 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":90,"threads_started":1}
I20260923 23:14:21.678462 25998 heartbeater.cc:347] Connected to a master server at 127.24.227.254:42135
I20260923 23:14:21.678535 25998 heartbeater.cc:464] Registering TS with master...
I20260923 23:14:21.678658 25998 heartbeater.cc:511] Master 127.24.227.254:42135 requested a full tablet report, sending...
I20260923 23:14:21.678952 25539 ts_manager.cc:194] Registered new tserver with Master: 2917076af3c14aa6b51d4b1e706443a8 (127.24.227.195:43191)
I20260923 23:14:21.679327 25539 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.227.195@KRBTEST.COM'} at 127.24.227.195:42907
I20260923 23:14:21.683302 25487 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 23 23:14:21 dist-test-slave-754n krb5kdc[25494](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205261, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 23 23:14:21 dist-test-slave-754n krb5kdc[25494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205261, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.24.227.193@KRBTEST.COM
I20260923 23:14:21.704866 26005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:21.697738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56798 (local address 127.24.227.193:39885)
0923 23:14:21.697904 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:21.697908 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:21.697918 (+ 10us) server_negotiation.cc:407] Connection header received
0923 23:14:21.697941 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:21.697942 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:21.697955 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:21.698018 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:21.698643 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:21.699364 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:21.700157 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:21.700473 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:21.700531 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:21.702531 (+ 2000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:21.702538 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:21.702538 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:21.702544 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:21.703828 (+ 1284us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:21.704191 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:21.704194 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:21.704194 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:21.704245 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:21.704398 (+ 153us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:21.704399 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:21.704399 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:21.704528 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:21.704622 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:21.704686 (+ 64us) server_negotiation.cc:299] Negotiation successful
0923 23:14:21.704743 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":94,"threads_started":1}
W20260923 23:14:21.705212 25644 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56798
Sep 23 23:14:21 dist-test-slave-754n krb5kdc[25494](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205261, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 23 23:14:21 dist-test-slave-754n krb5kdc[25494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205261, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.227.193@KRBTEST.COM
I20260923 23:14:21.725514 26005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:21.719259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56800 (local address 127.24.227.193:39885)
0923 23:14:21.719327 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:21.719329 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:21.719453 (+ 124us) server_negotiation.cc:407] Connection header received
0923 23:14:21.719489 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:21.719490 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:21.719500 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:21.719549 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:21.720115 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:21.720590 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:21.721235 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:21.721431 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:21.721477 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:21.723510 (+ 2033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:21.723511 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:21.723511 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:21.723517 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:21.724676 (+ 1159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:21.724951 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:21.724952 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:21.724952 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:21.724997 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:21.725132 (+ 135us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:21.725132 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:21.725133 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:21.725204 (+ 71us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:21.725287 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:21.725346 (+ 59us) server_negotiation.cc:299] Negotiation successful
0923 23:14:21.725418 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260923 23:14:21.726050 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 25601
I20260923 23:14:21.730283 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 25735
I20260923 23:14:21.734591 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 25869
I20260923 23:14:21.738610 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 25507
2026-09-23T23:14:21Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (1803 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790205259943518-25487-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 23:14:21 dist-test-slave-754n krb5kdc[26012](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 23:14:21 dist-test-slave-754n krb5kdc[26012](info): set up 2 sockets
Sep 23 23:14:21 dist-test-slave-754n krb5kdc[26012](info): commencing operation
krb5kdc: starting...
W20260923 23:14:22.743398 25487 mini_kdc.cc:121] Time spent starting KDC: real 0.985s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790205259943518-25487-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790205259943518-25487-0/krb5kdc/test-user.keytab.
Sep 23 23:14:22 dist-test-slave-754n krb5kdc[26012](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205262, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T23:14:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T23:14:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.227.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.227.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.227.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab.
Entry for principal kudu/127.24.227.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab.
I20260923 23:14:22.854106 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790205259943518-25487-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790205259943518-25487-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790205259943518-25487-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.254:35681
--webserver_interface=127.24.227.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab
--principal=kudu/127.24.227.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:45071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.227.254:35681
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:14:22.931190 26025 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:14:22.931356 26025 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:14:22.931376 26025 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:14:22.932487 26025 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 23:14:22.932528 26025 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:14:22.932541 26025 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:14:22.932552 26025 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 23:14:22.932601 26025 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 23:14:22.934007 26025 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:45071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790205259943518-25487-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790205259943518-25487-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.227.254:35681
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab
--principal=kudu/127.24.227.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.254:35681
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.227.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26025
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790205259943518-25487-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:14:22.934175 26025 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:14:22.934396 26025 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:14:22.938850 26030 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:22.938874 26031 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:22.938850 26033 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:14:22.939178 26025 server_base.cc:1034] running on GCE node
I20260923 23:14:22.939389 26025 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:14:22.939647 26025 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:14:22.940784 26025 hybrid_clock.cc:648] HybridClock initialized: now 1790205262940769 us; error 29 us; skew 500 ppm
Sep 23 23:14:22 dist-test-slave-754n krb5kdc[26012](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205262, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:14:22.953930 26025 init.cc:377] Logged in from keytab as kudu/127.24.227.254@KRBTEST.COM (short username kudu)
I20260923 23:14:22.954336 26025 webserver.cc:460] Webserver started at http://127.24.227.254:36061/ using document root <none> and password file <none>
I20260923 23:14:22.954573 26025 fs_manager.cc:359] Metadata directory not provided
I20260923 23:14:22.954640 26025 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:14:22.954785 26025 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:14:22.955781 26025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790205259943518-25487-0/minicluster-data/master-0/data/instance:
uuid: "79ee0ae5b3ef45f98c139ad1676e1b36"
format_stamp: "Formatted at 2026-09-23 23:14:22 on dist-test-slave-754n"
server_key: "4bf28fa0916c7162c7c5df40ca943c8f"
server_key_iv: "33cf5c82f1fe3ec8469b0d5c5284710b"
server_key_version: "encryptionkey@0"
I20260923 23:14:22.956192 26025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790205259943518-25487-0/minicluster-data/master-0/wal/instance:
uuid: "79ee0ae5b3ef45f98c139ad1676e1b36"
format_stamp: "Formatted at 2026-09-23 23:14:22 on dist-test-slave-754n"
server_key: "4bf28fa0916c7162c7c5df40ca943c8f"
server_key_iv: "33cf5c82f1fe3ec8469b0d5c5284710b"
server_key_version: "encryptionkey@0"
I20260923 23:14:22.957440 26025 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 23:14:22.958173 26040 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:22.958343 26025 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 23:14:22.958442 26025 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790205259943518-25487-0/minicluster-data/master-0/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790205259943518-25487-0/minicluster-data/master-0/wal
uuid: "79ee0ae5b3ef45f98c139ad1676e1b36"
format_stamp: "Formatted at 2026-09-23 23:14:22 on dist-test-slave-754n"
server_key: "4bf28fa0916c7162c7c5df40ca943c8f"
server_key_iv: "33cf5c82f1fe3ec8469b0d5c5284710b"
server_key_version: "encryptionkey@0"
I20260923 23:14:22.958511 26025 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790205259943518-25487-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790205259943518-25487-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790205259943518-25487-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:14:22.966532 26025 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:14:22.968923 26025 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:14:22.969079 26025 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:14:22.972916 26025 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.254:35681
I20260923 23:14:22.972944 26092 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.254:35681 every 8 connection(s)
I20260923 23:14:22.973415 26025 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
I20260923 23:14:22.974213 26093 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:14:22.976322 26093 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79ee0ae5b3ef45f98c139ad1676e1b36: Bootstrap starting.
I20260923 23:14:22.976804 26093 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 79ee0ae5b3ef45f98c139ad1676e1b36: Neither blocks nor log segments found. Creating new log.
I20260923 23:14:22.977025 26093 log.cc:824] T 00000000000000000000000000000000 P 79ee0ae5b3ef45f98c139ad1676e1b36: Log is configured to *not* fsync() on all Append() calls
I20260923 23:14:22.977665 26093 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79ee0ae5b3ef45f98c139ad1676e1b36: No bootstrap required, opened a new log
I20260923 23:14:22.977978 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 26025
I20260923 23:14:22.978075 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790205259943518-25487-0/minicluster-data/master-0/wal/instance
I20260923 23:14:22.978231 25487 external_mini_cluster.cc:1428] Setting key 61d8a58abb465b48edeff56ae0be16a5
I20260923 23:14:22.979794 26093 raft_consensus.cc:348] T 00000000000000000000000000000000 P 79ee0ae5b3ef45f98c139ad1676e1b36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79ee0ae5b3ef45f98c139ad1676e1b36" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 35681 } }
I20260923 23:14:22.979998 26093 raft_consensus.cc:374] T 00000000000000000000000000000000 P 79ee0ae5b3ef45f98c139ad1676e1b36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:14:22.980039 26093 raft_consensus.cc:729] T 00000000000000000000000000000000 P 79ee0ae5b3ef45f98c139ad1676e1b36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79ee0ae5b3ef45f98c139ad1676e1b36, State: Initialized, Role: FOLLOWER
I20260923 23:14:22.980170 26093 consensus_queue.cc:260] T 00000000000000000000000000000000 P 79ee0ae5b3ef45f98c139ad1676e1b36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79ee0ae5b3ef45f98c139ad1676e1b36" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 35681 } }
I20260923 23:14:22.980263 26093 raft_consensus.cc:388] T 00000000000000000000000000000000 P 79ee0ae5b3ef45f98c139ad1676e1b36 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 23:14:22.980296 26093 raft_consensus.cc:482] T 00000000000000000000000000000000 P 79ee0ae5b3ef45f98c139ad1676e1b36 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 23:14:22.980341 26093 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 79ee0ae5b3ef45f98c139ad1676e1b36 [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:14:22.980835 26093 raft_consensus.cc:504] T 00000000000000000000000000000000 P 79ee0ae5b3ef45f98c139ad1676e1b36 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79ee0ae5b3ef45f98c139ad1676e1b36" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 35681 } }
I20260923 23:14:22.980917 26093 leader_election.cc:302] T 00000000000000000000000000000000 P 79ee0ae5b3ef45f98c139ad1676e1b36 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 79ee0ae5b3ef45f98c139ad1676e1b36; no voters:
I20260923 23:14:22.981087 26093 leader_election.cc:288] T 00000000000000000000000000000000 P 79ee0ae5b3ef45f98c139ad1676e1b36 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 23:14:22.981127 26098 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 79ee0ae5b3ef45f98c139ad1676e1b36 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 23:14:22.981252 26098 raft_consensus.cc:686] T 00000000000000000000000000000000 P 79ee0ae5b3ef45f98c139ad1676e1b36 [term 1 LEADER]: Becoming Leader. State: Replica: 79ee0ae5b3ef45f98c139ad1676e1b36, State: Running, Role: LEADER
I20260923 23:14:22.981328 26098 consensus_queue.cc:237] T 00000000000000000000000000000000 P 79ee0ae5b3ef45f98c139ad1676e1b36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79ee0ae5b3ef45f98c139ad1676e1b36" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 35681 } }
I20260923 23:14:22.981354 26093 sys_catalog.cc:564] T 00000000000000000000000000000000 P 79ee0ae5b3ef45f98c139ad1676e1b36 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 23:14:22.981632 26098 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79ee0ae5b3ef45f98c139ad1676e1b36 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 79ee0ae5b3ef45f98c139ad1676e1b36. Latest consensus state: current_term: 1 leader_uuid: "79ee0ae5b3ef45f98c139ad1676e1b36" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79ee0ae5b3ef45f98c139ad1676e1b36" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 35681 } } }
I20260923 23:14:22.981709 26098 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79ee0ae5b3ef45f98c139ad1676e1b36 [sys.catalog]: This master's current role is: LEADER
I20260923 23:14:22.981642 26100 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79ee0ae5b3ef45f98c139ad1676e1b36 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "79ee0ae5b3ef45f98c139ad1676e1b36" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79ee0ae5b3ef45f98c139ad1676e1b36" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 35681 } } }
I20260923 23:14:22.981878 26100 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79ee0ae5b3ef45f98c139ad1676e1b36 [sys.catalog]: This master's current role is: LEADER
I20260923 23:14:22.982443 26103 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 23:14:22.982959 26103 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 23:14:22.984486 26103 catalog_manager.cc:1347] Generated new cluster ID: 0a09462305c9452198d36f3bccaba22b
I20260923 23:14:22.984539 26103 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 23:14:22 dist-test-slave-754n krb5kdc[26012](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205262, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
I20260923 23:14:22.989928 26097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:22.978717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51952 (local address 127.24.227.254:35681)
0923 23:14:22.978893 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:22.978905 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:22.978933 (+ 28us) server_negotiation.cc:407] Connection header received
0923 23:14:22.979332 (+ 399us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:22.979338 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:22.979379 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:22.979490 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:22.980275 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:22.981761 (+ 1486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:22.982731 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:22.983279 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:22.983986 (+ 707us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:22.986098 (+ 2112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:22.986102 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:22.986102 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:22.986113 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:22.988171 (+ 2058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:22.988590 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:22.988593 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:22.988594 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:22.988709 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:22.988916 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:22.988917 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:22.988917 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:22.989220 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:22.989410 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:22.989499 (+ 89us) server_negotiation.cc:299] Negotiation successful
0923 23:14:22.989734 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":90,"threads_started":1}
I20260923 23:14:23.001457 26103 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 23:14:23.002151 26103 catalog_manager.cc:1504] Loading token signing keys...
I20260923 23:14:23.005725 26103 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 79ee0ae5b3ef45f98c139ad1676e1b36: Generated new TSK 0
I20260923 23:14:23.005892 26103 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.227.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.227.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.227.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.193.keytab.
Entry for principal kudu/127.24.227.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.193.keytab.
I20260923 23:14:23.032845 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790205259943518-25487-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790205259943518-25487-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790205259943518-25487-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790205259943518-25487-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.227.193:0
--local_ip_for_outbound_sockets=127.24.227.193
--webserver_interface=127.24.227.193
--webserver_port=0
--tserver_master_addrs=127.24.227.254:35681
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.193.keytab
--principal=kudu/127.24.227.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:45071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:14:23.109669 26119 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:14:23.109894 26119 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:14:23.109933 26119 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:14:23.111039 26119 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:14:23.111106 26119 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:14:23.111151 26119 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.227.193
I20260923 23:14:23.112663 26119 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:45071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790205259943518-25487-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790205259943518-25487-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.193.keytab
--principal=kudu/127.24.227.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.227.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790205259943518-25487-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.227.193
--webserver_port=0
--tserver_master_addrs=127.24.227.254:35681
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26119
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.227.193
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790205259943518-25487-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:14:23.112915 26119 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:14:23.113137 26119 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:14:23.118026 26124 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:14:23.118031 26119 server_base.cc:1034] running on GCE node
W20260923 23:14:23.118003 26125 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:23.118000 26127 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:14:23.118474 26119 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:14:23.118741 26119 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:14:23.119963 26119 hybrid_clock.cc:648] HybridClock initialized: now 1790205263119953 us; error 42 us; skew 500 ppm
Sep 23 23:14:23 dist-test-slave-754n krb5kdc[26012](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205263, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:14:23.122418 26119 init.cc:377] Logged in from keytab as kudu/127.24.227.193@KRBTEST.COM (short username kudu)
I20260923 23:14:23.122777 26119 webserver.cc:460] Webserver started at http://127.24.227.193:45597/ using document root <none> and password file <none>
I20260923 23:14:23.122962 26119 fs_manager.cc:359] Metadata directory not provided
I20260923 23:14:23.123001 26119 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:14:23.123085 26119 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:14:23.124079 26119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790205259943518-25487-0/minicluster-data/ts-0/data/instance:
uuid: "3b56d640808743fb8ed797f17bbf5c95"
format_stamp: "Formatted at 2026-09-23 23:14:23 on dist-test-slave-754n"
server_key: "66e529a5098929b38660f59ef7aa0391"
server_key_iv: "a86642db57ab2d0ed8dcca33a1c5ca59"
server_key_version: "encryptionkey@0"
I20260923 23:14:23.124362 26119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790205259943518-25487-0/minicluster-data/ts-0/wal/instance:
uuid: "3b56d640808743fb8ed797f17bbf5c95"
format_stamp: "Formatted at 2026-09-23 23:14:23 on dist-test-slave-754n"
server_key: "66e529a5098929b38660f59ef7aa0391"
server_key_iv: "a86642db57ab2d0ed8dcca33a1c5ca59"
server_key_version: "encryptionkey@0"
I20260923 23:14:23.125499 26119 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 23:14:23.126108 26134 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:23.126288 26119 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 23:14:23.126384 26119 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790205259943518-25487-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790205259943518-25487-0/minicluster-data/ts-0/wal
uuid: "3b56d640808743fb8ed797f17bbf5c95"
format_stamp: "Formatted at 2026-09-23 23:14:23 on dist-test-slave-754n"
server_key: "66e529a5098929b38660f59ef7aa0391"
server_key_iv: "a86642db57ab2d0ed8dcca33a1c5ca59"
server_key_version: "encryptionkey@0"
I20260923 23:14:23.126434 26119 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790205259943518-25487-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790205259943518-25487-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790205259943518-25487-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:14:23.143784 26119 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:14:23.145932 26119 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:14:23.146052 26119 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:14:23.146210 26119 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 23:14:23.146425 26119 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 23:14:23.146452 26119 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:23.146472 26119 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 23:14:23.146484 26119 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:23.151859 26119 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.193:41887
I20260923 23:14:23.151954 26247 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.193:41887 every 8 connection(s)
I20260923 23:14:23.152264 26119 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790205259943518-25487-0/minicluster-data/ts-0/data/info.pb
I20260923 23:14:23.157140 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 26119
I20260923 23:14:23.157244 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790205259943518-25487-0/minicluster-data/ts-0/wal/instance
I20260923 23:14:23.157410 25487 external_mini_cluster.cc:1428] Setting key 4ccf038f23a30399ac4adfb4dd8029bb
Sep 23 23:14:23 dist-test-slave-754n krb5kdc[26012](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205263, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.193@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
I20260923 23:14:23.162076 26097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:23.152981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.193:46955 (local address 127.24.227.254:35681)
0923 23:14:23.153070 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:23.153072 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:23.153238 (+ 166us) server_negotiation.cc:407] Connection header received
0923 23:14:23.154133 (+ 895us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:23.154134 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:23.154146 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:23.154201 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:23.154790 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:23.155283 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:23.156276 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:23.156511 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:23.156595 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:23.159092 (+ 2497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:23.159093 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:23.159093 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:23.159098 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:23.160614 (+ 1516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:23.161115 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:23.161115 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:23.161116 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:23.161164 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:23.161449 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:23.161449 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:23.161449 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:23.161545 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:23.161594 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:23.161904 (+ 310us) server_negotiation.cc:299] Negotiation successful
0923 23:14:23.161968 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260923 23:14:23.162310 26250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:23.152948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:35681 (local address 127.24.227.193:46955)
0923 23:14:23.153123 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:23.153155 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:23.154022 (+ 867us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:23.154270 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:23.154273 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:23.154497 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:14:23.154683 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:23.154686 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:23.155396 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:23.155398 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:23.156185 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:23.156187 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:23.156607 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:23.156608 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:23.156724 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:23.157156 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:14:23.157166 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:14:23.158981 (+ 1815us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:14:23.160704 (+ 1723us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:23.160710 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:23.160712 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:23.161012 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:23.161233 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:23.161234 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:23.161234 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:23.161371 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:23.161611 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:14:23.161617 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:14:23.161771 (+ 154us) client_negotiation.cc:769] Sending connection context
0923 23:14:23.161885 (+ 114us) client_negotiation.cc:240] Negotiation successful
0923 23:14:23.162048 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":85,"threads_started":1}
I20260923 23:14:23.162889 26248 heartbeater.cc:347] Connected to a master server at 127.24.227.254:35681
I20260923 23:14:23.162974 26248 heartbeater.cc:464] Registering TS with master...
I20260923 23:14:23.163160 26248 heartbeater.cc:511] Master 127.24.227.254:35681 requested a full tablet report, sending...
I20260923 23:14:23.163633 26057 ts_manager.cc:194] Registered new tserver with Master: 3b56d640808743fb8ed797f17bbf5c95 (127.24.227.193:41887)
I20260923 23:14:23.164299 26057 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.227.193@KRBTEST.COM'} at 127.24.227.193:46955
WARNING: no policy specified for kudu/127.24.227.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.227.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.227.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.194.keytab.
Entry for principal kudu/127.24.227.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.194.keytab.
I20260923 23:14:23.180650 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790205259943518-25487-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790205259943518-25487-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790205259943518-25487-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790205259943518-25487-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.227.194:0
--local_ip_for_outbound_sockets=127.24.227.194
--webserver_interface=127.24.227.194
--webserver_port=0
--tserver_master_addrs=127.24.227.254:35681
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.194.keytab
--principal=kudu/127.24.227.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:45071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:14:23.258620 26253 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:14:23.258836 26253 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:14:23.258872 26253 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:14:23.260033 26253 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:14:23.260100 26253 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:14:23.260145 26253 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.227.194
I20260923 23:14:23.261622 26253 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:45071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790205259943518-25487-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790205259943518-25487-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.194.keytab
--principal=kudu/127.24.227.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.227.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790205259943518-25487-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.227.194
--webserver_port=0
--tserver_master_addrs=127.24.227.254:35681
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26253
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.227.194
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790205259943518-25487-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:14:23.261875 26253 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:14:23.262099 26253 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:14:23.266817 26258 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:23.266822 26259 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:23.266784 26261 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:14:23.266948 26253 server_base.cc:1034] running on GCE node
I20260923 23:14:23.267230 26253 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:14:23.267498 26253 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:14:23.268646 26253 hybrid_clock.cc:648] HybridClock initialized: now 1790205263268625 us; error 39 us; skew 500 ppm
Sep 23 23:14:23 dist-test-slave-754n krb5kdc[26012](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205263, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:14:23.271054 26253 init.cc:377] Logged in from keytab as kudu/127.24.227.194@KRBTEST.COM (short username kudu)
I20260923 23:14:23.271523 26253 webserver.cc:460] Webserver started at http://127.24.227.194:38403/ using document root <none> and password file <none>
I20260923 23:14:23.271698 26253 fs_manager.cc:359] Metadata directory not provided
I20260923 23:14:23.271737 26253 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:14:23.271821 26253 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:14:23.272817 26253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790205259943518-25487-0/minicluster-data/ts-1/data/instance:
uuid: "0518751b2c6c44c687b7ff2ff5c60aa4"
format_stamp: "Formatted at 2026-09-23 23:14:23 on dist-test-slave-754n"
server_key: "5706545084b7289e31a749ad7e7bea62"
server_key_iv: "51052eb0559f5656f201232cc8a24551"
server_key_version: "encryptionkey@0"
I20260923 23:14:23.273120 26253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790205259943518-25487-0/minicluster-data/ts-1/wal/instance:
uuid: "0518751b2c6c44c687b7ff2ff5c60aa4"
format_stamp: "Formatted at 2026-09-23 23:14:23 on dist-test-slave-754n"
server_key: "5706545084b7289e31a749ad7e7bea62"
server_key_iv: "51052eb0559f5656f201232cc8a24551"
server_key_version: "encryptionkey@0"
I20260923 23:14:23.274320 26253 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 23:14:23.274981 26268 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:23.275163 26253 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 23:14:23.275262 26253 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790205259943518-25487-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790205259943518-25487-0/minicluster-data/ts-1/wal
uuid: "0518751b2c6c44c687b7ff2ff5c60aa4"
format_stamp: "Formatted at 2026-09-23 23:14:23 on dist-test-slave-754n"
server_key: "5706545084b7289e31a749ad7e7bea62"
server_key_iv: "51052eb0559f5656f201232cc8a24551"
server_key_version: "encryptionkey@0"
I20260923 23:14:23.275338 26253 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790205259943518-25487-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790205259943518-25487-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790205259943518-25487-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:14:23.289674 26253 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:14:23.291424 26253 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:14:23.291594 26253 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:14:23.291796 26253 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 23:14:23.292124 26253 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 23:14:23.292181 26253 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:23.292220 26253 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 23:14:23.292248 26253 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:23.298276 26253 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.194:34813
I20260923 23:14:23.298316 26381 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.194:34813 every 8 connection(s)
I20260923 23:14:23.298619 26253 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790205259943518-25487-0/minicluster-data/ts-1/data/info.pb
Sep 23 23:14:23 dist-test-slave-754n krb5kdc[26012](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205263, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.194@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
I20260923 23:14:23.305013 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 26253
I20260923 23:14:23.305120 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790205259943518-25487-0/minicluster-data/ts-1/wal/instance
I20260923 23:14:23.305303 25487 external_mini_cluster.cc:1428] Setting key 7d2c7e7aae9d02b41b8d63875451c048
I20260923 23:14:23.308012 26097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:23.299342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.194:40845 (local address 127.24.227.254:35681)
0923 23:14:23.299385 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:23.299387 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:23.299591 (+ 204us) server_negotiation.cc:407] Connection header received
0923 23:14:23.300442 (+ 851us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:23.300443 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:23.300465 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:23.300520 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:23.301096 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:23.301562 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:23.302490 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:23.302682 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:23.302740 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:23.304784 (+ 2044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:23.304785 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:23.304785 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:23.304791 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:23.306508 (+ 1717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:23.306979 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:23.306981 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:23.306982 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:23.307046 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:23.307328 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:23.307329 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:23.307330 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:23.307449 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:23.307535 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:23.307816 (+ 281us) server_negotiation.cc:299] Negotiation successful
0923 23:14:23.307923 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 23:14:23.308200 26384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:23.299300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:35681 (local address 127.24.227.194:40845)
0923 23:14:23.299450 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:23.299499 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:23.300356 (+ 857us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:23.300581 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:23.300584 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:23.300833 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:14:23.301012 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:23.301016 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:23.301633 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:23.301633 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:23.302410 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:23.302412 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:23.302759 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:23.302760 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:23.302856 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:23.303393 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:14:23.303401 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:14:23.304676 (+ 1275us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:14:23.306683 (+ 2007us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:23.306689 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:23.306691 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:23.306895 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:23.307113 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:23.307114 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:23.307114 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:23.307253 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:23.307527 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:14:23.307533 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:14:23.307661 (+ 128us) client_negotiation.cc:769] Sending connection context
0923 23:14:23.307794 (+ 133us) client_negotiation.cc:240] Negotiation successful
0923 23:14:23.307989 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":86,"threads_started":1}
I20260923 23:14:23.308653 26382 heartbeater.cc:347] Connected to a master server at 127.24.227.254:35681
I20260923 23:14:23.308748 26382 heartbeater.cc:464] Registering TS with master...
I20260923 23:14:23.308924 26382 heartbeater.cc:511] Master 127.24.227.254:35681 requested a full tablet report, sending...
I20260923 23:14:23.309270 26057 ts_manager.cc:194] Registered new tserver with Master: 0518751b2c6c44c687b7ff2ff5c60aa4 (127.24.227.194:34813)
I20260923 23:14:23.309645 26057 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.227.194@KRBTEST.COM'} at 127.24.227.194:40845
WARNING: no policy specified for kudu/127.24.227.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.227.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.227.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.195.keytab.
Entry for principal kudu/127.24.227.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.195.keytab.
I20260923 23:14:23.328122 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790205259943518-25487-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790205259943518-25487-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790205259943518-25487-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790205259943518-25487-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.227.195:0
--local_ip_for_outbound_sockets=127.24.227.195
--webserver_interface=127.24.227.195
--webserver_port=0
--tserver_master_addrs=127.24.227.254:35681
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.195.keytab
--principal=kudu/127.24.227.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:45071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:14:23.405441 26387 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:14:23.405630 26387 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:14:23.405652 26387 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:14:23.406715 26387 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:14:23.406749 26387 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:14:23.406775 26387 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.227.195
I20260923 23:14:23.408156 26387 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:45071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790205259943518-25487-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790205259943518-25487-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.195.keytab
--principal=kudu/127.24.227.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.227.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790205259943518-25487-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.227.195
--webserver_port=0
--tserver_master_addrs=127.24.227.254:35681
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26387
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.227.195
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790205259943518-25487-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:14:23.408319 26387 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:14:23.408479 26387 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:14:23.413097 26393 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:23.413137 26395 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:23.413092 26392 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:14:23.413323 26387 server_base.cc:1034] running on GCE node
I20260923 23:14:23.413527 26387 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:14:23.413767 26387 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:14:23.414916 26387 hybrid_clock.cc:648] HybridClock initialized: now 1790205263414899 us; error 31 us; skew 500 ppm
Sep 23 23:14:23 dist-test-slave-754n krb5kdc[26012](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205263, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:14:23.417287 26387 init.cc:377] Logged in from keytab as kudu/127.24.227.195@KRBTEST.COM (short username kudu)
I20260923 23:14:23.417743 26387 webserver.cc:460] Webserver started at http://127.24.227.195:42191/ using document root <none> and password file <none>
I20260923 23:14:23.417992 26387 fs_manager.cc:359] Metadata directory not provided
I20260923 23:14:23.418037 26387 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:14:23.418195 26387 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:14:23.419199 26387 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790205259943518-25487-0/minicluster-data/ts-2/data/instance:
uuid: "8c91f7c83bb34f809c994244c1420775"
format_stamp: "Formatted at 2026-09-23 23:14:23 on dist-test-slave-754n"
server_key: "1d86ae036865fee3c641ced0f5975145"
server_key_iv: "933d08ed84df362096ab386b5ff275a7"
server_key_version: "encryptionkey@0"
I20260923 23:14:23.419559 26387 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790205259943518-25487-0/minicluster-data/ts-2/wal/instance:
uuid: "8c91f7c83bb34f809c994244c1420775"
format_stamp: "Formatted at 2026-09-23 23:14:23 on dist-test-slave-754n"
server_key: "1d86ae036865fee3c641ced0f5975145"
server_key_iv: "933d08ed84df362096ab386b5ff275a7"
server_key_version: "encryptionkey@0"
I20260923 23:14:23.420864 26387 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 23:14:23.421584 26402 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:23.421765 26387 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 23:14:23.421841 26387 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790205259943518-25487-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790205259943518-25487-0/minicluster-data/ts-2/wal
uuid: "8c91f7c83bb34f809c994244c1420775"
format_stamp: "Formatted at 2026-09-23 23:14:23 on dist-test-slave-754n"
server_key: "1d86ae036865fee3c641ced0f5975145"
server_key_iv: "933d08ed84df362096ab386b5ff275a7"
server_key_version: "encryptionkey@0"
I20260923 23:14:23.421891 26387 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790205259943518-25487-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790205259943518-25487-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790205259943518-25487-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:14:23.432647 26387 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:14:23.434724 26387 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:14:23.434828 26387 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:14:23.435024 26387 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 23:14:23.435251 26387 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 23:14:23.435281 26387 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:23.435302 26387 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 23:14:23.435313 26387 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:23.440807 26387 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.195:36401
I20260923 23:14:23.440894 26515 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.195:36401 every 8 connection(s)
I20260923 23:14:23.441156 26387 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790205259943518-25487-0/minicluster-data/ts-2/data/info.pb
I20260923 23:14:23.442159 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 26387
I20260923 23:14:23.442269 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790205259943518-25487-0/minicluster-data/ts-2/wal/instance
I20260923 23:14:23.442441 25487 external_mini_cluster.cc:1428] Setting key 37ac8429424fd4c9ec6be4fadfbd7b6f
Sep 23 23:14:23 dist-test-slave-754n krb5kdc[26012](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205263, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.195@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
I20260923 23:14:23.449644 26097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:23.441773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.195:35491 (local address 127.24.227.254:35681)
0923 23:14:23.441811 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:23.441813 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:23.442020 (+ 207us) server_negotiation.cc:407] Connection header received
0923 23:14:23.443012 (+ 992us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:23.443014 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:23.443025 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:23.443091 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:23.443704 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:23.444225 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:23.445115 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:23.445313 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:23.445390 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:23.447177 (+ 1787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:23.447179 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:23.447179 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:23.447185 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:23.448353 (+ 1168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:23.448756 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:23.448757 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:23.448758 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:23.448816 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:23.449074 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:23.449075 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:23.449075 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:23.449175 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:23.449241 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:23.449482 (+ 241us) server_negotiation.cc:299] Negotiation successful
0923 23:14:23.449563 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260923 23:14:23.449807 26518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:23.441746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:35681 (local address 127.24.227.195:35491)
0923 23:14:23.441902 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:23.441938 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:23.442909 (+ 971us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:23.443154 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:23.443158 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:23.443371 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:14:23.443617 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:23.443622 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:23.444308 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:23.444309 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:23.445030 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:23.445032 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:23.445398 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:23.445399 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:23.445526 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:23.445861 (+ 335us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:14:23.445868 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:14:23.447083 (+ 1215us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:14:23.448447 (+ 1364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:23.448453 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:23.448455 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:23.448668 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:23.448881 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:23.448882 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:23.448883 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:23.448987 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:23.449233 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:14:23.449239 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:14:23.449361 (+ 122us) client_negotiation.cc:769] Sending connection context
0923 23:14:23.449452 (+ 91us) client_negotiation.cc:240] Negotiation successful
0923 23:14:23.449584 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":95,"threads_started":1}
I20260923 23:14:23.450389 26516 heartbeater.cc:347] Connected to a master server at 127.24.227.254:35681
I20260923 23:14:23.450474 26516 heartbeater.cc:464] Registering TS with master...
I20260923 23:14:23.450632 26516 heartbeater.cc:511] Master 127.24.227.254:35681 requested a full tablet report, sending...
I20260923 23:14:23.450946 26057 ts_manager.cc:194] Registered new tserver with Master: 8c91f7c83bb34f809c994244c1420775 (127.24.227.195:36401)
I20260923 23:14:23.451279 26057 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.227.195@KRBTEST.COM'} at 127.24.227.195:35491
I20260923 23:14:23.453853 25487 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 23:14:23.461182 26097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:23.455303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51968 (local address 127.24.227.254:35681)
0923 23:14:23.455412 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:23.455414 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:23.455526 (+ 112us) server_negotiation.cc:407] Connection header received
0923 23:14:23.455592 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:23.455593 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:23.455602 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:23.455655 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:23.456196 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:23.456632 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:23.457296 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:23.457502 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:23.457579 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:23.458794 (+ 1215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:23.458796 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:23.458796 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:23.458803 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:23.459961 (+ 1158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:23.460318 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:23.460319 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:23.460320 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:23.460369 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:23.460619 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:23.460620 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:23.460620 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:23.460751 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:23.460856 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:23.461023 (+ 167us) server_negotiation.cc:299] Negotiation successful
0923 23:14:23.461093 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260923 23:14:23.462844 26057 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51968:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 23:14:23.464351 26057 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 23:14:23.470392 26525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:23.466389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.193:41887 (local address 127.0.0.1:41520)
0923 23:14:23.466551 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:23.466560 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:23.466600 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:23.466930 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:23.466931 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:23.466933 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:14:23.466976 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:23.466977 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:23.468215 (+ 1238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:23.468216 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:23.469158 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:23.469160 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:23.470138 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:23.470140 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:23.470208 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:23.470220 (+ 12us) client_negotiation.cc:769] Sending connection context
0923 23:14:23.470287 (+ 67us) client_negotiation.cc:240] Negotiation successful
0923 23:14:23.470331 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":74,"threads_started":1}
I20260923 23:14:23.470412 26529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:23.466621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41520 (local address 127.24.227.193:41887)
0923 23:14:23.466796 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:23.466800 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:23.466809 (+ 9us) server_negotiation.cc:407] Connection header received
0923 23:14:23.466844 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:23.466846 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:23.466859 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:23.466929 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:14:23.467031 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:23.468125 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:23.469250 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:23.470064 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:23.470128 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:23.470175 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:23.470307 (+ 132us) server_negotiation.cc:299] Negotiation successful
0923 23:14:23.470342 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":86,"threads_started":1}
I20260923 23:14:23.470834 26524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:23.466217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.194:34813 (local address 127.0.0.1:42752)
0923 23:14:23.466360 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:23.466393 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:23.466451 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:23.466804 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:23.466807 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:23.466809 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:14:23.466865 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:23.466867 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:23.469071 (+ 2204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:23.469072 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:23.469816 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:23.469819 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:23.470624 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:23.470624 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:23.470686 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:23.470689 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 23:14:23.470736 (+ 47us) client_negotiation.cc:240] Negotiation successful
0923 23:14:23.470774 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":51,"threads_started":1}
I20260923 23:14:23.470868 26527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:23.466310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42752 (local address 127.24.227.194:34813)
0923 23:14:23.466650 (+ 340us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:23.466654 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:23.466663 (+ 9us) server_negotiation.cc:407] Connection header received
0923 23:14:23.466700 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:23.466702 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:23.466720 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:23.466800 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:14:23.467059 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:23.468935 (+ 1876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:23.469917 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:23.470568 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:23.470626 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:23.470670 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:23.470761 (+ 91us) server_negotiation.cc:299] Negotiation successful
0923 23:14:23.470798 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":70,"threads_started":1}
I20260923 23:14:23.471148 26182 tablet_service.cc:1467] Processing CreateTablet for tablet 0ae2fbdf426c4a8aa53a25c7f2015eaf (DEFAULT_TABLE table=test-table [id=24ed5e0090f2487b865a7a30284917b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 23:14:23.471408 26528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:23.466217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.195:36401 (local address 127.0.0.1:50978)
0923 23:14:23.466526 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:23.466535 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:23.466594 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:23.466838 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:23.466838 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:23.466840 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:14:23.466873 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:23.466874 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:23.468792 (+ 1918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:23.468793 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:23.469855 (+ 1062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:23.469857 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:23.471165 (+ 1308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:23.471167 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:23.471224 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:23.471228 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 23:14:23.471287 (+ 59us) client_negotiation.cc:240] Negotiation successful
0923 23:14:23.471347 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"mutex_wait_us":2,"thread_start_us":54,"threads_started":1}
I20260923 23:14:23.471490 26182 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ae2fbdf426c4a8aa53a25c7f2015eaf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:14:23.471516 26526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:23.466291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50978 (local address 127.24.227.195:36401)
0923 23:14:23.466493 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:23.466497 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:23.466650 (+ 153us) server_negotiation.cc:407] Connection header received
0923 23:14:23.466690 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:23.466692 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:23.466706 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:23.466772 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:14:23.467948 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:23.468698 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:23.469956 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:23.470736 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:23.470803 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:23.470845 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:23.471392 (+ 547us) server_negotiation.cc:299] Negotiation successful
0923 23:14:23.471428 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":75,"threads_started":1}
I20260923 23:14:23.471539 26316 tablet_service.cc:1467] Processing CreateTablet for tablet 0ae2fbdf426c4a8aa53a25c7f2015eaf (DEFAULT_TABLE table=test-table [id=24ed5e0090f2487b865a7a30284917b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 23:14:23.471814 26316 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ae2fbdf426c4a8aa53a25c7f2015eaf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:14:23.474007 26530 tablet_bootstrap.cc:492] T 0ae2fbdf426c4a8aa53a25c7f2015eaf P 3b56d640808743fb8ed797f17bbf5c95: Bootstrap starting.
I20260923 23:14:23.474156 26531 tablet_bootstrap.cc:492] T 0ae2fbdf426c4a8aa53a25c7f2015eaf P 0518751b2c6c44c687b7ff2ff5c60aa4: Bootstrap starting.
I20260923 23:14:23.474689 26530 tablet_bootstrap.cc:654] T 0ae2fbdf426c4a8aa53a25c7f2015eaf P 3b56d640808743fb8ed797f17bbf5c95: Neither blocks nor log segments found. Creating new log.
I20260923 23:14:23.474587 26450 tablet_service.cc:1467] Processing CreateTablet for tablet 0ae2fbdf426c4a8aa53a25c7f2015eaf (DEFAULT_TABLE table=test-table [id=24ed5e0090f2487b865a7a30284917b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 23:14:23.474684 26531 tablet_bootstrap.cc:654] T 0ae2fbdf426c4a8aa53a25c7f2015eaf P 0518751b2c6c44c687b7ff2ff5c60aa4: Neither blocks nor log segments found. Creating new log.
I20260923 23:14:23.474879 26450 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ae2fbdf426c4a8aa53a25c7f2015eaf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:14:23.475179 26531 log.cc:824] T 0ae2fbdf426c4a8aa53a25c7f2015eaf P 0518751b2c6c44c687b7ff2ff5c60aa4: Log is configured to *not* fsync() on all Append() calls
I20260923 23:14:23.475314 26530 log.cc:824] T 0ae2fbdf426c4a8aa53a25c7f2015eaf P 3b56d640808743fb8ed797f17bbf5c95: Log is configured to *not* fsync() on all Append() calls
I20260923 23:14:23.476071 26530 tablet_bootstrap.cc:492] T 0ae2fbdf426c4a8aa53a25c7f2015eaf P 3b56d640808743fb8ed797f17bbf5c95: No bootstrap required, opened a new log
I20260923 23:14:23.476141 26530 ts_tablet_manager.cc:1397] T 0ae2fbdf426c4a8aa53a25c7f2015eaf P 3b56d640808743fb8ed797f17bbf5c95: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 23:14:23.476855 26531 tablet_bootstrap.cc:492] T 0ae2fbdf426c4a8aa53a25c7f2015eaf P 0518751b2c6c44c687b7ff2ff5c60aa4: No bootstrap required, opened a new log
I20260923 23:14:23.476935 26531 ts_tablet_manager.cc:1397] T 0ae2fbdf426c4a8aa53a25c7f2015eaf P 0518751b2c6c44c687b7ff2ff5c60aa4: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260923 23:14:23.478246 26530 raft_consensus.cc:348] T 0ae2fbdf426c4a8aa53a25c7f2015eaf P 3b56d640808743fb8ed797f17bbf5c95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c91f7c83bb34f809c994244c1420775" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 36401 } } peers { permanent_uuid: "3b56d640808743fb8ed797f17bbf5c95" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 41887 } } peers { permanent_uuid: "0518751b2c6c44c687b7ff2ff5c60aa4" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 34813 } }
I20260923 23:14:23.478422 26530 raft_consensus.cc:374] T 0ae2fbdf426c4a8aa53a25c7f2015eaf P 3b56d640808743fb8ed797f17bbf5c95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:14:23.478467 26530 raft_consensus.cc:729] T 0ae2fbdf426c4a8aa53a25c7f2015eaf P 3b56d640808743fb8ed797f17bbf5c95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b56d640808743fb8ed797f17bbf5c95, State: Initialized, Role: FOLLOWER
I20260923 23:14:23.478570 26530 consensus_queue.cc:260] T 0ae2fbdf426c4a8aa53a25c7f2015eaf P 3b56d640808743fb8ed797f17bbf5c95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c91f7c83bb34f809c994244c1420775" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 36401 } } peers { permanent_uuid: "3b56d640808743fb8ed797f17bbf5c95" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 41887 } } peers { permanent_uuid: "0518751b2c6c44c687b7ff2ff5c60aa4" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 34813 } }
I20260923 23:14:23.478578 26531 raft_consensus.cc:348] T 0ae2fbdf426c4a8aa53a25c7f2015eaf P 0518751b2c6c44c687b7ff2ff5c60aa4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c91f7c83bb34f809c994244c1420775" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 36401 } } peers { permanent_uuid: "3b56d640808743fb8ed797f17bbf5c95" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 41887 } } peers { permanent_uuid: "0518751b2c6c44c687b7ff2ff5c60aa4" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 34813 } }
I20260923 23:14:23.478780 26531 raft_consensus.cc:374] T 0ae2fbdf426c4a8aa53a25c7f2015eaf P 0518751b2c6c44c687b7ff2ff5c60aa4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:14:23.478825 26531 raft_consensus.cc:729] T 0ae2fbdf426c4a8aa53a25c7f2015eaf P 0518751b2c6c44c687b7ff2ff5c60aa4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0518751b2c6c44c687b7ff2ff5c60aa4, State: Initialized, Role: FOLLOWER
I20260923 23:14:23.478852 26530 ts_tablet_manager.cc:1428] T 0ae2fbdf426c4a8aa53a25c7f2015eaf P 3b56d640808743fb8ed797f17bbf5c95: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260923 23:14:23.478967 26248 heartbeater.cc:503] Master 127.24.227.254:35681 was elected leader, sending a full tablet report...
I20260923 23:14:23.478940 26531 consensus_queue.cc:260] T 0ae2fbdf426c4a8aa53a25c7f2015eaf P 0518751b2c6c44c687b7ff2ff5c60aa4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c91f7c83bb34f809c994244c1420775" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 36401 } } peers { permanent_uuid: "3b56d640808743fb8ed797f17bbf5c95" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 41887 } } peers { permanent_uuid: "0518751b2c6c44c687b7ff2ff5c60aa4" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 34813 } }
I20260923 23:14:23.479066 26534 tablet_bootstrap.cc:492] T 0ae2fbdf426c4a8aa53a25c7f2015eaf P 8c91f7c83bb34f809c994244c1420775: Bootstrap starting.
I20260923 23:14:23.479200 26531 ts_tablet_manager.cc:1428] T 0ae2fbdf426c4a8aa53a25c7f2015eaf P 0518751b2c6c44c687b7ff2ff5c60aa4: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260923 23:14:23.479315 26382 heartbeater.cc:503] Master 127.24.227.254:35681 was elected leader, sending a full tablet report...
I20260923 23:14:23.479665 26534 tablet_bootstrap.cc:654] T 0ae2fbdf426c4a8aa53a25c7f2015eaf P 8c91f7c83bb34f809c994244c1420775: Neither blocks nor log segments found. Creating new log.
I20260923 23:14:23.480244 26534 log.cc:824] T 0ae2fbdf426c4a8aa53a25c7f2015eaf P 8c91f7c83bb34f809c994244c1420775: Log is configured to *not* fsync() on all Append() calls
I20260923 23:14:23.481197 26536 raft_consensus.cc:482] T 0ae2fbdf426c4a8aa53a25c7f2015eaf P 0518751b2c6c44c687b7ff2ff5c60aa4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 23:14:23.481268 26536 raft_consensus.cc:504] T 0ae2fbdf426c4a8aa53a25c7f2015eaf P 0518751b2c6c44c687b7ff2ff5c60aa4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c91f7c83bb34f809c994244c1420775" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 36401 } } peers { permanent_uuid: "3b56d640808743fb8ed797f17bbf5c95" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 41887 } } peers { permanent_uuid: "0518751b2c6c44c687b7ff2ff5c60aa4" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 34813 } }
I20260923 23:14:23.481591 26536 leader_election.cc:288] T 0ae2fbdf426c4a8aa53a25c7f2015eaf P 0518751b2c6c44c687b7ff2ff5c60aa4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c91f7c83bb34f809c994244c1420775 (127.24.227.195:36401), 3b56d640808743fb8ed797f17bbf5c95 (127.24.227.193:41887)
I20260923 23:14:23.483911 26534 tablet_bootstrap.cc:492] T 0ae2fbdf426c4a8aa53a25c7f2015eaf P 8c91f7c83bb34f809c994244c1420775: No bootstrap required, opened a new log
I20260923 23:14:23.483976 26534 ts_tablet_manager.cc:1397] T 0ae2fbdf426c4a8aa53a25c7f2015eaf P 8c91f7c83bb34f809c994244c1420775: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260923 23:14:23.485474 26384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:23.481744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.195:36401 (local address 127.24.227.194:45209)
0923 23:14:23.481991 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:23.482001 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:23.482061 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:23.482270 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:23.482270 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:23.482272 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:14:23.482323 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:23.482328 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:23.483203 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:23.483204 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:23.484574 (+ 1370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:23.484577 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:23.485246 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:23.485247 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:23.485324 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:23.485326 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 23:14:23.485369 (+ 43us) client_negotiation.cc:240] Negotiation successful
0923 23:14:23.485403 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232}
I20260923 23:14:23.485697 26526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:23.481759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.194:45209 (local address 127.24.227.195:36401)
0923 23:14:23.481805 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:23.481807 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:23.482167 (+ 360us) server_negotiation.cc:407] Connection header received
0923 23:14:23.482191 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:23.482191 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:23.482200 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:23.482251 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:14:23.482430 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:23.483105 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:23.484669 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:23.485141 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:23.485210 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:23.485217 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:23.485614 (+ 397us) server_negotiation.cc:299] Negotiation successful
0923 23:14:23.485647 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260923 23:14:23.485920 26470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ae2fbdf426c4a8aa53a25c7f2015eaf" candidate_uuid: "0518751b2c6c44c687b7ff2ff5c60aa4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c91f7c83bb34f809c994244c1420775" is_pre_election: true
I20260923 23:14:23.486310 26534 raft_consensus.cc:348] T 0ae2fbdf426c4a8aa53a25c7f2015eaf P 8c91f7c83bb34f809c994244c1420775 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c91f7c83bb34f809c994244c1420775" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 36401 } } peers { permanent_uuid: "3b56d640808743fb8ed797f17bbf5c95" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 41887 } } peers { permanent_uuid: "0518751b2c6c44c687b7ff2ff5c60aa4" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 34813 } }
I20260923 23:14:23.486418 26534 raft_consensus.cc:374] T 0ae2fbdf426c4a8aa53a25c7f2015eaf P 8c91f7c83bb34f809c994244c1420775 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:14:23.486442 26534 raft_consensus.cc:729] T 0ae2fbdf426c4a8aa53a25c7f2015eaf P 8c91f7c83bb34f809c994244c1420775 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c91f7c83bb34f809c994244c1420775, State: Initialized, Role: FOLLOWER
W20260923 23:14:23.486537 26269 leader_election.cc:341] T 0ae2fbdf426c4a8aa53a25c7f2015eaf P 0518751b2c6c44c687b7ff2ff5c60aa4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c91f7c83bb34f809c994244c1420775 (127.24.227.195:36401): Illegal state: must be running to vote when last-logged opid is not known
I20260923 23:14:23.486539 26534 consensus_queue.cc:260] T 0ae2fbdf426c4a8aa53a25c7f2015eaf P 8c91f7c83bb34f809c994244c1420775 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c91f7c83bb34f809c994244c1420775" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 36401 } } peers { permanent_uuid: "3b56d640808743fb8ed797f17bbf5c95" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 41887 } } peers { permanent_uuid: "0518751b2c6c44c687b7ff2ff5c60aa4" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 34813 } }
I20260923 23:14:23.486765 26516 heartbeater.cc:503] Master 127.24.227.254:35681 was elected leader, sending a full tablet report...
I20260923 23:14:23.486775 26534 ts_tablet_manager.cc:1428] T 0ae2fbdf426c4a8aa53a25c7f2015eaf P 8c91f7c83bb34f809c994244c1420775: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260923 23:14:23.487582 26529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:23.482184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.194:36487 (local address 127.24.227.193:41887)
0923 23:14:23.482961 (+ 777us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:23.482963 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:23.482971 (+ 8us) server_negotiation.cc:407] Connection header received
0923 23:14:23.482988 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:23.482989 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:23.482996 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:23.483055 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:14:23.484780 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:23.485436 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:23.486654 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:23.487205 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:23.487280 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:23.487287 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:23.487471 (+ 184us) server_negotiation.cc:299] Negotiation successful
0923 23:14:23.487508 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":758}
I20260923 23:14:23.487586 26538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:23.481854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.193:41887 (local address 127.24.227.194:36487)
0923 23:14:23.481997 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:23.482007 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:23.482084 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:23.484627 (+ 2543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:23.484628 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:23.484630 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:14:23.484694 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:23.484696 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:23.485543 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:23.485544 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:23.486549 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:23.486551 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:23.487339 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:23.487340 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:23.487403 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:23.487407 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 23:14:23.487464 (+ 57us) client_negotiation.cc:240] Negotiation successful
0923 23:14:23.487502 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":99,"threads_started":1}
I20260923 23:14:23.487877 26202 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ae2fbdf426c4a8aa53a25c7f2015eaf" candidate_uuid: "0518751b2c6c44c687b7ff2ff5c60aa4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b56d640808743fb8ed797f17bbf5c95" is_pre_election: true
I20260923 23:14:23.488013 26202 raft_consensus.cc:2454] T 0ae2fbdf426c4a8aa53a25c7f2015eaf P 3b56d640808743fb8ed797f17bbf5c95 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0518751b2c6c44c687b7ff2ff5c60aa4 in term 0.
I20260923 23:14:23.488219 26269 leader_election.cc:302] T 0ae2fbdf426c4a8aa53a25c7f2015eaf P 0518751b2c6c44c687b7ff2ff5c60aa4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0518751b2c6c44c687b7ff2ff5c60aa4, 3b56d640808743fb8ed797f17bbf5c95; no voters: 8c91f7c83bb34f809c994244c1420775
I20260923 23:14:23.488333 26536 raft_consensus.cc:2781] T 0ae2fbdf426c4a8aa53a25c7f2015eaf P 0518751b2c6c44c687b7ff2ff5c60aa4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 23:14:23.488374 26536 raft_consensus.cc:482] T 0ae2fbdf426c4a8aa53a25c7f2015eaf P 0518751b2c6c44c687b7ff2ff5c60aa4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 23:14:23.488396 26536 raft_consensus.cc:3037] T 0ae2fbdf426c4a8aa53a25c7f2015eaf P 0518751b2c6c44c687b7ff2ff5c60aa4 [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:14:23.488965 26536 raft_consensus.cc:504] T 0ae2fbdf426c4a8aa53a25c7f2015eaf P 0518751b2c6c44c687b7ff2ff5c60aa4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c91f7c83bb34f809c994244c1420775" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 36401 } } peers { permanent_uuid: "3b56d640808743fb8ed797f17bbf5c95" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 41887 } } peers { permanent_uuid: "0518751b2c6c44c687b7ff2ff5c60aa4" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 34813 } }
I20260923 23:14:23.489099 26536 leader_election.cc:288] T 0ae2fbdf426c4a8aa53a25c7f2015eaf P 0518751b2c6c44c687b7ff2ff5c60aa4 [CANDIDATE]: Term 1 election: Requested vote from peers 8c91f7c83bb34f809c994244c1420775 (127.24.227.195:36401), 3b56d640808743fb8ed797f17bbf5c95 (127.24.227.193:41887)
I20260923 23:14:23.489256 26202 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ae2fbdf426c4a8aa53a25c7f2015eaf" candidate_uuid: "0518751b2c6c44c687b7ff2ff5c60aa4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b56d640808743fb8ed797f17bbf5c95"
I20260923 23:14:23.489250 26470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ae2fbdf426c4a8aa53a25c7f2015eaf" candidate_uuid: "0518751b2c6c44c687b7ff2ff5c60aa4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c91f7c83bb34f809c994244c1420775"
I20260923 23:14:23.489321 26470 raft_consensus.cc:3037] T 0ae2fbdf426c4a8aa53a25c7f2015eaf P 8c91f7c83bb34f809c994244c1420775 [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:14:23.489318 26202 raft_consensus.cc:3037] T 0ae2fbdf426c4a8aa53a25c7f2015eaf P 3b56d640808743fb8ed797f17bbf5c95 [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:14:23.490058 26470 raft_consensus.cc:2454] T 0ae2fbdf426c4a8aa53a25c7f2015eaf P 8c91f7c83bb34f809c994244c1420775 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0518751b2c6c44c687b7ff2ff5c60aa4 in term 1.
I20260923 23:14:23.490165 26202 raft_consensus.cc:2454] T 0ae2fbdf426c4a8aa53a25c7f2015eaf P 3b56d640808743fb8ed797f17bbf5c95 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0518751b2c6c44c687b7ff2ff5c60aa4 in term 1.
I20260923 23:14:23.490229 26269 leader_election.cc:302] T 0ae2fbdf426c4a8aa53a25c7f2015eaf P 0518751b2c6c44c687b7ff2ff5c60aa4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0518751b2c6c44c687b7ff2ff5c60aa4, 8c91f7c83bb34f809c994244c1420775; no voters:
I20260923 23:14:23.490336 26536 raft_consensus.cc:2781] T 0ae2fbdf426c4a8aa53a25c7f2015eaf P 0518751b2c6c44c687b7ff2ff5c60aa4 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 23:14:23.490468 26536 raft_consensus.cc:686] T 0ae2fbdf426c4a8aa53a25c7f2015eaf P 0518751b2c6c44c687b7ff2ff5c60aa4 [term 1 LEADER]: Becoming Leader. State: Replica: 0518751b2c6c44c687b7ff2ff5c60aa4, State: Running, Role: LEADER
I20260923 23:14:23.490556 26536 consensus_queue.cc:237] T 0ae2fbdf426c4a8aa53a25c7f2015eaf P 0518751b2c6c44c687b7ff2ff5c60aa4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c91f7c83bb34f809c994244c1420775" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 36401 } } peers { permanent_uuid: "3b56d640808743fb8ed797f17bbf5c95" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 41887 } } peers { permanent_uuid: "0518751b2c6c44c687b7ff2ff5c60aa4" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 34813 } }
I20260923 23:14:23.491209 26056 catalog_manager.cc:5495] T 0ae2fbdf426c4a8aa53a25c7f2015eaf P 0518751b2c6c44c687b7ff2ff5c60aa4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0518751b2c6c44c687b7ff2ff5c60aa4 (127.24.227.194). New cstate: current_term: 1 leader_uuid: "0518751b2c6c44c687b7ff2ff5c60aa4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c91f7c83bb34f809c994244c1420775" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 36401 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b56d640808743fb8ed797f17bbf5c95" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 41887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0518751b2c6c44c687b7ff2ff5c60aa4" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 34813 } health_report { overall_health: HEALTHY } } }
Sep 23 23:14:23 dist-test-slave-754n krb5kdc[26012](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205262, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.227.193@KRBTEST.COM
I20260923 23:14:23.503106 26529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:23.496719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41536 (local address 127.24.227.193:41887)
0923 23:14:23.496780 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:23.496782 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:23.496885 (+ 103us) server_negotiation.cc:407] Connection header received
0923 23:14:23.496922 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:23.496922 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:23.496930 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:23.496973 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:23.497430 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:23.497870 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:23.498582 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:23.498750 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:23.498797 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:23.500930 (+ 2133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:23.500936 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:23.500937 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:23.500944 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:23.502257 (+ 1313us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:23.502558 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:23.502560 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:23.502561 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:23.502611 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:23.502758 (+ 147us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:23.502758 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:23.502759 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:23.502864 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:23.502953 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:23.503000 (+ 47us) server_negotiation.cc:299] Negotiation successful
0923 23:14:23.503052 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
Sep 23 23:14:23 dist-test-slave-754n krb5kdc[26012](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205263, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 23 23:14:23 dist-test-slave-754n krb5kdc[26012](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205263, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.24.227.193@KRBTEST.COM
I20260923 23:14:23.523957 26529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:23.517444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41542 (local address 127.24.227.193:41887)
0923 23:14:23.517510 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:23.517512 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:23.517559 (+ 47us) server_negotiation.cc:407] Connection header received
0923 23:14:23.517601 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:23.517602 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:23.517610 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:23.517656 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:23.518195 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:23.518642 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:23.519295 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:23.519469 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:23.519519 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:23.521567 (+ 2048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:23.521568 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:23.521568 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:23.521574 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:23.522816 (+ 1242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:23.523127 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:23.523128 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:23.523128 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:23.523176 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:23.523399 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:23.523400 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:23.523400 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:23.523481 (+ 81us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:23.523550 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:23.523728 (+ 178us) server_negotiation.cc:299] Negotiation successful
0923 23:14:23.523822 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
Sep 23 23:14:23 dist-test-slave-754n krb5kdc[26012](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205263, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 23 23:14:23 dist-test-slave-754n krb5kdc[26012](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205263, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.227.193@KRBTEST.COM
I20260923 23:14:23.544418 26529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:23.537842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41548 (local address 127.24.227.193:41887)
0923 23:14:23.537903 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:23.537905 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:23.537999 (+ 94us) server_negotiation.cc:407] Connection header received
0923 23:14:23.538079 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:23.538080 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:23.538096 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:23.538159 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:23.538672 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:23.539120 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:23.539816 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:23.540085 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:23.540154 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:23.542181 (+ 2027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:23.542183 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:23.542183 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:23.542190 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:23.543290 (+ 1100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:23.543646 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:23.543647 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:23.543648 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:23.543706 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:23.543944 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:23.543945 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:23.543945 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:23.544043 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:23.544159 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:23.544256 (+ 97us) server_negotiation.cc:299] Negotiation successful
0923 23:14:23.544322 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
W20260923 23:14:23.544873 26162 tablet_service.cc:2908] Rejecting scan request for tablet 0ae2fbdf426c4a8aa53a25c7f2015eaf: Uninitialized: clean time has not yet been initialized
I20260923 23:14:23.545663 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 26119
W20260923 23:14:23.549146 26383 tablet.cc:2369] T 0ae2fbdf426c4a8aa53a25c7f2015eaf P 0518751b2c6c44c687b7ff2ff5c60aa4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 23:14:23.550217 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 26253
I20260923 23:14:23.554881 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 26387
I20260923 23:14:23.559124 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 26025
2026-09-23T23:14:23Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (1820 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 23:14:23 dist-test-slave-754n krb5kdc[26554](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 23:14:23 dist-test-slave-754n krb5kdc[26554](info): set up 2 sockets
Sep 23 23:14:23 dist-test-slave-754n krb5kdc[26554](info): commencing operation
krb5kdc: starting...
W20260923 23:14:24.551192 25487 mini_kdc.cc:121] Time spent starting KDC: real 0.972s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/krb5kdc/test-user.keytab.
Sep 23 23:14:24 dist-test-slave-754n krb5kdc[26554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205264, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T23:14:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T23:14:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.227.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.227.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.227.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab.
Entry for principal kudu/127.24.227.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab.
I20260923 23:14:24.656967 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.254:41471
--webserver_interface=127.24.227.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab
--principal=kudu/127.24.227.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:38931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.227.254:41471
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:14:24.730965 26567 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:14:24.731174 26567 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:14:24.731206 26567 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:14:24.732304 26567 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 23:14:24.732368 26567 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260923 23:14:24.732388 26567 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:14:24.732409 26567 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:14:24.732438 26567 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 23:14:24.732461 26567 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 23:14:24.733767 26567 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:38931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.227.254:41471
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab
--principal=kudu/127.24.227.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.254:41471
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.227.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26567
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:14:24.734036 26567 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:14:24.734254 26567 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:14:24.738713 26575 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:24.738739 26573 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:14:24.738976 26567 server_base.cc:1034] running on GCE node
W20260923 23:14:24.738724 26572 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:14:24.739265 26567 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:14:24.739550 26567 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:14:24.740717 26567 hybrid_clock.cc:648] HybridClock initialized: now 1790205264740705 us; error 25 us; skew 500 ppm
Sep 23 23:14:24 dist-test-slave-754n krb5kdc[26554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205264, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:14:24.743134 26567 init.cc:377] Logged in from keytab as kudu/127.24.227.254@KRBTEST.COM (short username kudu)
I20260923 23:14:24.743533 26567 webserver.cc:460] Webserver started at http://127.24.227.254:43323/ using document root <none> and password file <none>
I20260923 23:14:24.743752 26567 fs_manager.cc:359] Metadata directory not provided
I20260923 23:14:24.743816 26567 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:14:24.743983 26567 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:14:24.744973 26567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/minicluster-data/master-0/data/instance:
uuid: "edb11fcb0fbd4223a79b05d5bfd4e8da"
format_stamp: "Formatted at 2026-09-23 23:14:24 on dist-test-slave-754n"
server_key: "c6f6673f96f7e00d07c4d035dde7c216"
server_key_iv: "a59c1ce5d2d763d576219b199acdfd9e"
server_key_version: "encryptionkey@0"
I20260923 23:14:24.745316 26567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/minicluster-data/master-0/wal/instance:
uuid: "edb11fcb0fbd4223a79b05d5bfd4e8da"
format_stamp: "Formatted at 2026-09-23 23:14:24 on dist-test-slave-754n"
server_key: "c6f6673f96f7e00d07c4d035dde7c216"
server_key_iv: "a59c1ce5d2d763d576219b199acdfd9e"
server_key_version: "encryptionkey@0"
I20260923 23:14:24.746580 26567 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260923 23:14:24.747241 26582 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:24.747403 26567 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 23:14:24.747498 26567 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/minicluster-data/master-0/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/minicluster-data/master-0/wal
uuid: "edb11fcb0fbd4223a79b05d5bfd4e8da"
format_stamp: "Formatted at 2026-09-23 23:14:24 on dist-test-slave-754n"
server_key: "c6f6673f96f7e00d07c4d035dde7c216"
server_key_iv: "a59c1ce5d2d763d576219b199acdfd9e"
server_key_version: "encryptionkey@0"
I20260923 23:14:24.747562 26567 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:14:24.758961 26567 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:14:24.761299 26567 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:14:24.761463 26567 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:14:24.765893 26567 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.254:41471
I20260923 23:14:24.765962 26644 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.254:41471 every 8 connection(s)
I20260923 23:14:24.766252 26567 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
I20260923 23:14:24.766814 26645 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:14:24.769064 26645 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P edb11fcb0fbd4223a79b05d5bfd4e8da: Bootstrap starting.
I20260923 23:14:24.769649 26645 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P edb11fcb0fbd4223a79b05d5bfd4e8da: Neither blocks nor log segments found. Creating new log.
I20260923 23:14:24.769919 26645 log.cc:824] T 00000000000000000000000000000000 P edb11fcb0fbd4223a79b05d5bfd4e8da: Log is configured to *not* fsync() on all Append() calls
I20260923 23:14:24.770591 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 26567
I20260923 23:14:24.770622 26645 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P edb11fcb0fbd4223a79b05d5bfd4e8da: No bootstrap required, opened a new log
I20260923 23:14:24.770666 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/minicluster-data/master-0/wal/instance
I20260923 23:14:24.770823 25487 external_mini_cluster.cc:1428] Setting key ecdc4d15bcddca272deefa1ff7cde83c
I20260923 23:14:24.773454 26645 raft_consensus.cc:348] T 00000000000000000000000000000000 P edb11fcb0fbd4223a79b05d5bfd4e8da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edb11fcb0fbd4223a79b05d5bfd4e8da" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41471 } }
I20260923 23:14:24.773617 26645 raft_consensus.cc:374] T 00000000000000000000000000000000 P edb11fcb0fbd4223a79b05d5bfd4e8da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:14:24.773650 26645 raft_consensus.cc:729] T 00000000000000000000000000000000 P edb11fcb0fbd4223a79b05d5bfd4e8da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: edb11fcb0fbd4223a79b05d5bfd4e8da, State: Initialized, Role: FOLLOWER
I20260923 23:14:24.773758 26645 consensus_queue.cc:260] T 00000000000000000000000000000000 P edb11fcb0fbd4223a79b05d5bfd4e8da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edb11fcb0fbd4223a79b05d5bfd4e8da" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41471 } }
I20260923 23:14:24.773821 26645 raft_consensus.cc:388] T 00000000000000000000000000000000 P edb11fcb0fbd4223a79b05d5bfd4e8da [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 23:14:24.773849 26645 raft_consensus.cc:482] T 00000000000000000000000000000000 P edb11fcb0fbd4223a79b05d5bfd4e8da [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 23:14:24.773883 26645 raft_consensus.cc:3037] T 00000000000000000000000000000000 P edb11fcb0fbd4223a79b05d5bfd4e8da [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:14:24.774468 26645 raft_consensus.cc:504] T 00000000000000000000000000000000 P edb11fcb0fbd4223a79b05d5bfd4e8da [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edb11fcb0fbd4223a79b05d5bfd4e8da" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41471 } }
I20260923 23:14:24.774554 26645 leader_election.cc:302] T 00000000000000000000000000000000 P edb11fcb0fbd4223a79b05d5bfd4e8da [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: edb11fcb0fbd4223a79b05d5bfd4e8da; no voters:
I20260923 23:14:24.774745 26645 leader_election.cc:288] T 00000000000000000000000000000000 P edb11fcb0fbd4223a79b05d5bfd4e8da [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 23:14:24.774771 26650 raft_consensus.cc:2781] T 00000000000000000000000000000000 P edb11fcb0fbd4223a79b05d5bfd4e8da [term 1 FOLLOWER]: Leader election won for term 1
I20260923 23:14:24.774893 26650 raft_consensus.cc:686] T 00000000000000000000000000000000 P edb11fcb0fbd4223a79b05d5bfd4e8da [term 1 LEADER]: Becoming Leader. State: Replica: edb11fcb0fbd4223a79b05d5bfd4e8da, State: Running, Role: LEADER
I20260923 23:14:24.774972 26650 consensus_queue.cc:237] T 00000000000000000000000000000000 P edb11fcb0fbd4223a79b05d5bfd4e8da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edb11fcb0fbd4223a79b05d5bfd4e8da" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41471 } }
I20260923 23:14:24.775003 26645 sys_catalog.cc:564] T 00000000000000000000000000000000 P edb11fcb0fbd4223a79b05d5bfd4e8da [sys.catalog]: configured and running, proceeding with master startup.
I20260923 23:14:24.775305 26650 sys_catalog.cc:455] T 00000000000000000000000000000000 P edb11fcb0fbd4223a79b05d5bfd4e8da [sys.catalog]: SysCatalogTable state changed. Reason: New leader edb11fcb0fbd4223a79b05d5bfd4e8da. Latest consensus state: current_term: 1 leader_uuid: "edb11fcb0fbd4223a79b05d5bfd4e8da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edb11fcb0fbd4223a79b05d5bfd4e8da" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41471 } } }
I20260923 23:14:24.775314 26651 sys_catalog.cc:455] T 00000000000000000000000000000000 P edb11fcb0fbd4223a79b05d5bfd4e8da [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "edb11fcb0fbd4223a79b05d5bfd4e8da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edb11fcb0fbd4223a79b05d5bfd4e8da" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41471 } } }
I20260923 23:14:24.775390 26650 sys_catalog.cc:458] T 00000000000000000000000000000000 P edb11fcb0fbd4223a79b05d5bfd4e8da [sys.catalog]: This master's current role is: LEADER
I20260923 23:14:24.775418 26651 sys_catalog.cc:458] T 00000000000000000000000000000000 P edb11fcb0fbd4223a79b05d5bfd4e8da [sys.catalog]: This master's current role is: LEADER
I20260923 23:14:24.776309 26656 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 23:14:24.777518 26656 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 23 23:14:24 dist-test-slave-754n krb5kdc[26554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205264, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
I20260923 23:14:24.779110 26656 catalog_manager.cc:1347] Generated new cluster ID: d1b7ae7a9eac4ee795db2c0c4f7ed6ff
I20260923 23:14:24.779160 26656 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 23:14:24.781893 26649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:24.771300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52914 (local address 127.24.227.254:41471)
0923 23:14:24.771496 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:24.771505 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:24.771538 (+ 33us) server_negotiation.cc:407] Connection header received
0923 23:14:24.772041 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:24.772049 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:24.772108 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:24.772246 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:24.772798 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:24.774110 (+ 1312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:24.774975 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:24.775543 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:24.775625 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:24.778324 (+ 2699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:24.778327 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:24.778328 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:24.778339 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:24.780310 (+ 1971us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:24.780679 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:24.780682 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:24.780683 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:24.780786 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:24.780991 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:24.780992 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:24.780992 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:24.781285 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:24.781439 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:24.781526 (+ 87us) server_negotiation.cc:299] Negotiation successful
0923 23:14:24.781669 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":98,"threads_started":1}
I20260923 23:14:24.790232 26656 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 23:14:24.790889 26656 catalog_manager.cc:1504] Loading token signing keys...
I20260923 23:14:24.794844 26656 catalog_manager.cc:5868] T 00000000000000000000000000000000 P edb11fcb0fbd4223a79b05d5bfd4e8da: Generated new TSK 0
I20260923 23:14:24.795004 26656 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.227.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.227.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.227.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.193.keytab.
Entry for principal kudu/127.24.227.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.193.keytab.
I20260923 23:14:24.822201 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.227.193:0
--local_ip_for_outbound_sockets=127.24.227.193
--webserver_interface=127.24.227.193
--webserver_port=0
--tserver_master_addrs=127.24.227.254:41471
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.193.keytab
--principal=kudu/127.24.227.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:38931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:14:24.902668 26671 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:14:24.902848 26671 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:14:24.902869 26671 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:14:24.904057 26671 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:14:24.904093 26671 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:14:24.904112 26671 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260923 23:14:24.904147 26671 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.227.193
I20260923 23:14:24.905491 26671 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:38931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.193.keytab
--principal=kudu/127.24.227.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.227.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.227.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.227.254:41471
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26671
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.227.193
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:14:24.905714 26671 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:14:24.905941 26671 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:14:24.911749 26679 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:24.911748 26676 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:24.911841 26677 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:14:24.911939 26671 server_base.cc:1034] running on GCE node
I20260923 23:14:24.912109 26671 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:14:24.912386 26671 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:14:24.913532 26671 hybrid_clock.cc:648] HybridClock initialized: now 1790205264913521 us; error 27 us; skew 500 ppm
Sep 23 23:14:24 dist-test-slave-754n krb5kdc[26554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205264, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:14:24.916010 26671 init.cc:377] Logged in from keytab as kudu/127.24.227.193@KRBTEST.COM (short username kudu)
I20260923 23:14:24.916465 26671 webserver.cc:460] Webserver started at http://127.24.227.193:43551/ using document root <none> and password file <none>
I20260923 23:14:24.916680 26671 fs_manager.cc:359] Metadata directory not provided
I20260923 23:14:24.916744 26671 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:14:24.916874 26671 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:14:24.917893 26671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/minicluster-data/ts-0/data/instance:
uuid: "b92ee785a3c04711abeb66c9af684149"
format_stamp: "Formatted at 2026-09-23 23:14:24 on dist-test-slave-754n"
server_key: "9fcdc5924b735436b3f0356f6284e7ec"
server_key_iv: "0398a1f5bc869f82970e7ecd6993cd94"
server_key_version: "encryptionkey@0"
I20260923 23:14:24.918249 26671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/minicluster-data/ts-0/wal/instance:
uuid: "b92ee785a3c04711abeb66c9af684149"
format_stamp: "Formatted at 2026-09-23 23:14:24 on dist-test-slave-754n"
server_key: "9fcdc5924b735436b3f0356f6284e7ec"
server_key_iv: "0398a1f5bc869f82970e7ecd6993cd94"
server_key_version: "encryptionkey@0"
I20260923 23:14:24.919561 26671 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 23:14:24.920351 26686 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:24.920539 26671 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:24.920634 26671 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/minicluster-data/ts-0/wal
uuid: "b92ee785a3c04711abeb66c9af684149"
format_stamp: "Formatted at 2026-09-23 23:14:24 on dist-test-slave-754n"
server_key: "9fcdc5924b735436b3f0356f6284e7ec"
server_key_iv: "0398a1f5bc869f82970e7ecd6993cd94"
server_key_version: "encryptionkey@0"
I20260923 23:14:24.920728 26671 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:14:24.947955 26671 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:14:24.949826 26671 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:14:24.949985 26671 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:14:24.950520 26671 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 23:14:24.950575 26671 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:24.950613 26671 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 23:14:24.950630 26671 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 23 23:14:24 dist-test-slave-754n krb5kdc[26554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205264, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.193@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
I20260923 23:14:24.957935 26671 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.193:36649
I20260923 23:14:24.958019 26801 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.193:36649 every 8 connection(s)
I20260923 23:14:24.958355 26671 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/minicluster-data/ts-0/data/info.pb
I20260923 23:14:24.960317 26649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:24.950861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.193:48159 (local address 127.24.227.254:41471)
0923 23:14:24.950908 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:24.950911 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:24.951203 (+ 292us) server_negotiation.cc:407] Connection header received
0923 23:14:24.951733 (+ 530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:24.951734 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:24.951746 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:24.951806 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:24.952506 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:24.953033 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:24.954000 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:24.954329 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:24.954405 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:24.957603 (+ 3198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:24.957605 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:24.957605 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:24.957613 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:24.959183 (+ 1570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:24.959549 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:24.959549 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:24.959550 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:24.959598 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:24.959796 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:24.959797 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:24.959797 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:24.959948 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:24.960009 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:24.960197 (+ 188us) server_negotiation.cc:299] Negotiation successful
0923 23:14:24.960256 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260923 23:14:24.960494 26695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:24.950856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41471 (local address 127.24.227.193:48159)
0923 23:14:24.951022 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:24.951058 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:24.951603 (+ 545us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:24.951928 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:24.951932 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:24.952226 (+ 294us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:14:24.952409 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:24.952413 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:24.953128 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:24.953130 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:24.953898 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:24.953900 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:24.954423 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:24.954424 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:24.954594 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:24.955178 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:14:24.955188 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:14:24.957503 (+ 2315us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:14:24.959256 (+ 1753us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:24.959260 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:24.959262 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:24.959469 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:24.959654 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:24.959654 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:24.959655 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:24.959743 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:24.960000 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:14:24.960003 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:14:24.960104 (+ 101us) client_negotiation.cc:769] Sending connection context
0923 23:14:24.960186 (+ 82us) client_negotiation.cc:240] Negotiation successful
0923 23:14:24.960295 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":108,"threads_started":1}
I20260923 23:14:24.961010 26802 heartbeater.cc:347] Connected to a master server at 127.24.227.254:41471
I20260923 23:14:24.961086 26802 heartbeater.cc:464] Registering TS with master...
I20260923 23:14:24.961251 26802 heartbeater.cc:511] Master 127.24.227.254:41471 requested a full tablet report, sending...
I20260923 23:14:24.961798 26599 ts_manager.cc:194] Registered new tserver with Master: b92ee785a3c04711abeb66c9af684149 (127.24.227.193:36649)
I20260923 23:14:24.962769 26599 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.227.193@KRBTEST.COM'} at 127.24.227.193:48159
I20260923 23:14:24.966848 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 26671
I20260923 23:14:24.966953 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/minicluster-data/ts-0/wal/instance
I20260923 23:14:24.967079 26649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:24.962003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.193:43553 (local address 127.24.227.254:41471)
0923 23:14:24.962051 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:24.962053 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:24.962194 (+ 141us) server_negotiation.cc:407] Connection header received
0923 23:14:24.962217 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:24.962218 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:24.962227 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:24.962287 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:24.962570 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:24.963173 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:24.963954 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:24.964134 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:24.964191 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:24.965043 (+ 852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:24.965044 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:24.965045 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:24.965049 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:24.966081 (+ 1032us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:24.966410 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:24.966411 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:24.966412 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:24.966462 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:24.966625 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:24.966626 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:24.966627 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:24.966726 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:24.966792 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:24.966937 (+ 145us) server_negotiation.cc:299] Negotiation successful
0923 23:14:24.966997 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260923 23:14:24.967092 26808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:24.962002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41471 (local address 127.24.227.193:43553)
0923 23:14:24.962125 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:24.962134 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:24.962182 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:24.962290 (+ 108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:24.962290 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:24.962437 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:14:24.962485 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:24.962486 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:24.963256 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:24.963256 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:24.963804 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:24.963806 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:24.964198 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:24.964199 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:24.964256 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:24.964670 (+ 414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:14:24.964673 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:14:24.964975 (+ 302us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:14:24.966177 (+ 1202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:24.966178 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:24.966178 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:24.966333 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:24.966513 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:24.966514 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:24.966514 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:24.966561 (+ 47us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:24.966794 (+ 233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:14:24.966795 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:14:24.966834 (+ 39us) client_negotiation.cc:769] Sending connection context
0923 23:14:24.966913 (+ 79us) client_negotiation.cc:240] Negotiation successful
0923 23:14:24.967000 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"mutex_wait_us":30,"thread_start_us":71,"threads_started":1}
I20260923 23:14:24.967130 25487 external_mini_cluster.cc:1428] Setting key b5e7efb861597e1c99da1f4548aecdc6
WARNING: no policy specified for kudu/127.24.227.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.227.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.227.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.194.keytab.
Entry for principal kudu/127.24.227.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.194.keytab.
I20260923 23:14:24.991040 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.227.194:0
--local_ip_for_outbound_sockets=127.24.227.194
--webserver_interface=127.24.227.194
--webserver_port=0
--tserver_master_addrs=127.24.227.254:41471
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.194.keytab
--principal=kudu/127.24.227.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:38931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:14:25.068049 26811 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:14:25.068265 26811 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:14:25.068322 26811 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:14:25.069499 26811 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:14:25.069577 26811 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:14:25.069610 26811 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260923 23:14:25.069645 26811 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.227.194
I20260923 23:14:25.070990 26811 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:38931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.194.keytab
--principal=kudu/127.24.227.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.227.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.227.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.227.254:41471
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26811
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.227.194
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:14:25.071237 26811 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:14:25.071463 26811 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:14:25.076104 26819 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:25.076181 26817 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:25.076118 26816 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:14:25.076330 26811 server_base.cc:1034] running on GCE node
I20260923 23:14:25.076540 26811 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:14:25.076752 26811 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:14:25.077906 26811 hybrid_clock.cc:648] HybridClock initialized: now 1790205265077890 us; error 32 us; skew 500 ppm
Sep 23 23:14:25 dist-test-slave-754n krb5kdc[26554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205265, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:14:25.080341 26811 init.cc:377] Logged in from keytab as kudu/127.24.227.194@KRBTEST.COM (short username kudu)
I20260923 23:14:25.080801 26811 webserver.cc:460] Webserver started at http://127.24.227.194:41113/ using document root <none> and password file <none>
I20260923 23:14:25.081027 26811 fs_manager.cc:359] Metadata directory not provided
I20260923 23:14:25.081092 26811 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:14:25.081238 26811 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:14:25.082185 26811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/minicluster-data/ts-1/data/instance:
uuid: "08c54376cc9d473596a9b8c014946779"
format_stamp: "Formatted at 2026-09-23 23:14:25 on dist-test-slave-754n"
server_key: "641774d41dda2d8e1b662c83b1d7047c"
server_key_iv: "b54a932436d55ff88b0eca4ab3ca33fc"
server_key_version: "encryptionkey@0"
I20260923 23:14:25.082528 26811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/minicluster-data/ts-1/wal/instance:
uuid: "08c54376cc9d473596a9b8c014946779"
format_stamp: "Formatted at 2026-09-23 23:14:25 on dist-test-slave-754n"
server_key: "641774d41dda2d8e1b662c83b1d7047c"
server_key_iv: "b54a932436d55ff88b0eca4ab3ca33fc"
server_key_version: "encryptionkey@0"
I20260923 23:14:25.083758 26811 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260923 23:14:25.084429 26826 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:25.084586 26811 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:25.084674 26811 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/minicluster-data/ts-1/wal
uuid: "08c54376cc9d473596a9b8c014946779"
format_stamp: "Formatted at 2026-09-23 23:14:25 on dist-test-slave-754n"
server_key: "641774d41dda2d8e1b662c83b1d7047c"
server_key_iv: "b54a932436d55ff88b0eca4ab3ca33fc"
server_key_version: "encryptionkey@0"
I20260923 23:14:25.084754 26811 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:14:25.092219 26811 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:14:25.094333 26811 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:14:25.094491 26811 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:14:25.094954 26811 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 23:14:25.095017 26811 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:25.095074 26811 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 23:14:25.095093 26811 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 23 23:14:25 dist-test-slave-754n krb5kdc[26554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205265, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.194@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
I20260923 23:14:25.102156 26811 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.194:44665
I20260923 23:14:25.102274 26941 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.194:44665 every 8 connection(s)
I20260923 23:14:25.102545 26811 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/minicluster-data/ts-1/data/info.pb
I20260923 23:14:25.103761 26649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:25.095306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.194:44085 (local address 127.24.227.254:41471)
0923 23:14:25.095345 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:25.095347 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:25.095541 (+ 194us) server_negotiation.cc:407] Connection header received
0923 23:14:25.095997 (+ 456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:25.095999 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:25.096014 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:25.096094 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:25.096632 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:25.097135 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:25.098193 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:25.098574 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:25.098649 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:25.101002 (+ 2353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:25.101005 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:25.101005 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:25.101014 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:25.102567 (+ 1553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:25.103016 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:25.103017 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:25.103017 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:25.103080 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:25.103286 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:25.103286 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:25.103286 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:25.103376 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:25.103427 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:25.103644 (+ 217us) server_negotiation.cc:299] Negotiation successful
0923 23:14:25.103699 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260923 23:14:25.103996 26835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:25.095309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41471 (local address 127.24.227.194:44085)
0923 23:14:25.095430 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:25.095465 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:25.095843 (+ 378us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:25.096139 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:25.096142 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:25.096357 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:14:25.096529 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:25.096532 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:25.097231 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:25.097233 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:25.098116 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:25.098118 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:25.098680 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:25.098681 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:25.098789 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:25.099239 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:14:25.099247 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:14:25.100893 (+ 1646us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:14:25.102657 (+ 1764us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:25.102663 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:25.102665 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:25.102928 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:25.103143 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:25.103144 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:25.103144 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:25.103245 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:25.103419 (+ 174us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:14:25.103422 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:14:25.103519 (+ 97us) client_negotiation.cc:769] Sending connection context
0923 23:14:25.103631 (+ 112us) client_negotiation.cc:240] Negotiation successful
0923 23:14:25.103768 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":66,"threads_started":1}
I20260923 23:14:25.104470 26942 heartbeater.cc:347] Connected to a master server at 127.24.227.254:41471
I20260923 23:14:25.104544 26942 heartbeater.cc:464] Registering TS with master...
I20260923 23:14:25.104722 26942 heartbeater.cc:511] Master 127.24.227.254:41471 requested a full tablet report, sending...
I20260923 23:14:25.105026 26599 ts_manager.cc:194] Registered new tserver with Master: 08c54376cc9d473596a9b8c014946779 (127.24.227.194:44665)
I20260923 23:14:25.105233 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 26811
I20260923 23:14:25.105309 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/minicluster-data/ts-1/wal/instance
I20260923 23:14:25.105468 25487 external_mini_cluster.cc:1428] Setting key 4e3d5efe37f007a4314c06a99bfd2e56
I20260923 23:14:25.105517 26599 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.227.194@KRBTEST.COM'} at 127.24.227.194:44085
I20260923 23:14:25.110973 26649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:25.105499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.194:44011 (local address 127.24.227.254:41471)
0923 23:14:25.105532 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:25.105534 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:25.105662 (+ 128us) server_negotiation.cc:407] Connection header received
0923 23:14:25.105741 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:25.105743 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:25.105752 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:25.105801 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:25.106130 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:25.106566 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:25.107332 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:25.107538 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:25.107587 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:25.108485 (+ 898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:25.108486 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:25.108487 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:25.108492 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:25.109830 (+ 1338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:25.110174 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:25.110175 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:25.110175 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:25.110227 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:25.110426 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:25.110427 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:25.110428 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:25.110574 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:25.110648 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:25.110808 (+ 160us) server_negotiation.cc:299] Negotiation successful
0923 23:14:25.110889 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260923 23:14:25.110999 26948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:25.105424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41471 (local address 127.24.227.194:44011)
0923 23:14:25.105599 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:25.105611 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:25.105685 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:25.105814 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:25.105814 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:25.105998 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:14:25.106066 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:25.106068 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:25.106656 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:25.106657 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:25.107249 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:25.107252 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:25.107606 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:25.107607 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:25.107682 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:25.108102 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:14:25.108108 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:14:25.108384 (+ 276us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:14:25.109919 (+ 1535us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:25.109920 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:25.109920 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:25.110097 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:25.110290 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:25.110291 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:25.110292 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:25.110346 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:25.110642 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:14:25.110643 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:14:25.110696 (+ 53us) client_negotiation.cc:769] Sending connection context
0923 23:14:25.110782 (+ 86us) client_negotiation.cc:240] Negotiation successful
0923 23:14:25.110889 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":65,"threads_started":1}
WARNING: no policy specified for kudu/127.24.227.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.227.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.227.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.195.keytab.
Entry for principal kudu/127.24.227.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.195.keytab.
I20260923 23:14:25.129997 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.227.195:0
--local_ip_for_outbound_sockets=127.24.227.195
--webserver_interface=127.24.227.195
--webserver_port=0
--tserver_master_addrs=127.24.227.254:41471
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.195.keytab
--principal=kudu/127.24.227.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:38931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:14:25.205972 26951 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:14:25.206147 26951 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:14:25.206169 26951 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:14:25.207207 26951 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:14:25.207240 26951 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:14:25.207260 26951 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260923 23:14:25.207276 26951 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.227.195
I20260923 23:14:25.208587 26951 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:38931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.195.keytab
--principal=kudu/127.24.227.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.227.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.227.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.227.254:41471
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26951
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.227.195
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:14:25.208747 26951 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:14:25.208992 26951 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:14:25.213717 26956 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:25.213696 26957 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:25.213703 26959 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:14:25.213730 26951 server_base.cc:1034] running on GCE node
I20260923 23:14:25.214119 26951 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:14:25.214352 26951 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:14:25.215513 26951 hybrid_clock.cc:648] HybridClock initialized: now 1790205265215497 us; error 29 us; skew 500 ppm
Sep 23 23:14:25 dist-test-slave-754n krb5kdc[26554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205265, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:14:25.217886 26951 init.cc:377] Logged in from keytab as kudu/127.24.227.195@KRBTEST.COM (short username kudu)
I20260923 23:14:25.218329 26951 webserver.cc:460] Webserver started at http://127.24.227.195:41591/ using document root <none> and password file <none>
I20260923 23:14:25.218591 26951 fs_manager.cc:359] Metadata directory not provided
I20260923 23:14:25.218658 26951 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:14:25.218796 26951 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:14:25.219774 26951 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/minicluster-data/ts-2/data/instance:
uuid: "643a0fc32e094e1eb52717bd61a2fd85"
format_stamp: "Formatted at 2026-09-23 23:14:25 on dist-test-slave-754n"
server_key: "b53bae9cc2be3298d69e70144cb9babe"
server_key_iv: "3e0939c7e4e2b5db5805a10ab7bfbe62"
server_key_version: "encryptionkey@0"
I20260923 23:14:25.220158 26951 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/minicluster-data/ts-2/wal/instance:
uuid: "643a0fc32e094e1eb52717bd61a2fd85"
format_stamp: "Formatted at 2026-09-23 23:14:25 on dist-test-slave-754n"
server_key: "b53bae9cc2be3298d69e70144cb9babe"
server_key_iv: "3e0939c7e4e2b5db5805a10ab7bfbe62"
server_key_version: "encryptionkey@0"
I20260923 23:14:25.221424 26951 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 23:14:25.222179 26966 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:25.222368 26951 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 23:14:25.222464 26951 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/minicluster-data/ts-2/wal
uuid: "643a0fc32e094e1eb52717bd61a2fd85"
format_stamp: "Formatted at 2026-09-23 23:14:25 on dist-test-slave-754n"
server_key: "b53bae9cc2be3298d69e70144cb9babe"
server_key_iv: "3e0939c7e4e2b5db5805a10ab7bfbe62"
server_key_version: "encryptionkey@0"
I20260923 23:14:25.222535 26951 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:14:25.235383 26951 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:14:25.237082 26951 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:14:25.237258 26951 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:14:25.237775 26951 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 23:14:25.237831 26951 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:25.237874 26951 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 23:14:25.237903 26951 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 23 23:14:25 dist-test-slave-754n krb5kdc[26554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205265, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.195@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
I20260923 23:14:25.245025 26951 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.195:39787
I20260923 23:14:25.245121 27081 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.195:39787 every 8 connection(s)
I20260923 23:14:25.245383 26951 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/minicluster-data/ts-2/data/info.pb
I20260923 23:14:25.247562 26649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:25.238108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.195:37715 (local address 127.24.227.254:41471)
0923 23:14:25.238155 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:25.238158 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:25.238352 (+ 194us) server_negotiation.cc:407] Connection header received
0923 23:14:25.238856 (+ 504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:25.238857 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:25.238869 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:25.238920 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:25.239540 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:25.240113 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:25.241101 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:25.241290 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:25.241388 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:25.244664 (+ 3276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:25.244665 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:25.244666 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:25.244672 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:25.246172 (+ 1500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:25.246638 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:25.246640 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:25.246640 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:25.246707 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:25.246942 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:25.246943 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:25.246943 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:25.247059 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:25.247130 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:25.247398 (+ 268us) server_negotiation.cc:299] Negotiation successful
0923 23:14:25.247473 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 23:14:25.247740 26975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:25.238098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41471 (local address 127.24.227.195:37715)
0923 23:14:25.238229 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:25.238264 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:25.238757 (+ 493us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:25.238998 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:25.239001 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:25.239273 (+ 272us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:14:25.239440 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:25.239445 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:25.240201 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:25.240202 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:25.241002 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:25.241005 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:25.241397 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:25.241399 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:25.241518 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:25.242072 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:14:25.242083 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:14:25.244577 (+ 2494us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:14:25.246269 (+ 1692us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:25.246276 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:25.246278 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:25.246557 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:25.246773 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:25.246774 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:25.246774 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:25.246877 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:25.247122 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:14:25.247127 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:14:25.247263 (+ 136us) client_negotiation.cc:769] Sending connection context
0923 23:14:25.247362 (+ 99us) client_negotiation.cc:240] Negotiation successful
0923 23:14:25.247529 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":75,"threads_started":1}
I20260923 23:14:25.248199 27082 heartbeater.cc:347] Connected to a master server at 127.24.227.254:41471
I20260923 23:14:25.248274 27082 heartbeater.cc:464] Registering TS with master...
I20260923 23:14:25.248453 27082 heartbeater.cc:511] Master 127.24.227.254:41471 requested a full tablet report, sending...
I20260923 23:14:25.248842 26598 ts_manager.cc:194] Registered new tserver with Master: 643a0fc32e094e1eb52717bd61a2fd85 (127.24.227.195:39787)
I20260923 23:14:25.249356 26598 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.227.195@KRBTEST.COM'} at 127.24.227.195:37715
I20260923 23:14:25.253698 26649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:25.249199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.195:56065 (local address 127.24.227.254:41471)
0923 23:14:25.249258 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:25.249260 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:25.249388 (+ 128us) server_negotiation.cc:407] Connection header received
0923 23:14:25.249426 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:25.249427 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:25.249436 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:25.249483 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:25.249874 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:25.250305 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:25.250892 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:25.251072 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:25.251121 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:25.251804 (+ 683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:25.251805 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:25.251805 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:25.251810 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:25.252944 (+ 1134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:25.253233 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:25.253233 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:25.253233 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:25.253289 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:25.253397 (+ 108us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:25.253398 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:25.253398 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:25.253477 (+ 79us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:25.253519 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:25.253600 (+ 81us) server_negotiation.cc:299] Negotiation successful
0923 23:14:25.253648 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260923 23:14:25.253711 27088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:25.249167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41471 (local address 127.24.227.195:56065)
0923 23:14:25.249319 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:25.249328 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:25.249377 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:25.249522 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:25.249523 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:25.249731 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:14:25.249797 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:25.249799 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:25.250367 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:25.250368 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:25.250825 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:25.250826 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:25.251132 (+ 306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:25.251133 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:25.251182 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:25.251501 (+ 319us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:14:25.251504 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:14:25.251737 (+ 233us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:14:25.253012 (+ 1275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:25.253013 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:25.253013 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:25.253171 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:25.253327 (+ 156us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:25.253327 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:25.253328 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:25.253367 (+ 39us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:25.253510 (+ 143us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:14:25.253511 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:14:25.253541 (+ 30us) client_negotiation.cc:769] Sending connection context
0923 23:14:25.253584 (+ 43us) client_negotiation.cc:240] Negotiation successful
0923 23:14:25.253648 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":114,"threads_started":1}
I20260923 23:14:25.253921 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 26951
I20260923 23:14:25.254009 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/minicluster-data/ts-2/wal/instance
I20260923 23:14:25.254205 25487 external_mini_cluster.cc:1428] Setting key 9f1184b6e89418b2fcb45a3e66939094
I20260923 23:14:25.255136 25487 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 23 23:14:25 dist-test-slave-754n krb5kdc[26554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205265, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 23 23:14:25 dist-test-slave-754n krb5kdc[26554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205265, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
I20260923 23:14:25.275579 26649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:25.269426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52924 (local address 127.24.227.254:41471)
0923 23:14:25.269489 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:25.269491 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:25.269501 (+ 10us) server_negotiation.cc:407] Connection header received
0923 23:14:25.269525 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:25.269526 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:25.269534 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:25.269581 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:25.270091 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:25.270563 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:25.271204 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:25.271365 (+ 161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:25.271410 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:25.273487 (+ 2077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:25.273488 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:25.273488 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:25.273494 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:25.274674 (+ 1180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:25.274946 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:25.274946 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:25.274947 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:25.274993 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:25.275203 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:25.275204 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:25.275204 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:25.275291 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:25.275403 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:25.275472 (+ 69us) server_negotiation.cc:299] Negotiation successful
0923 23:14:25.275522 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260923 23:14:25.277040 26598 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52924:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 23:14:25.278530 26598 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 23:14:25.284680 27096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:25.280674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.194:44665 (local address 127.0.0.1:35486)
0923 23:14:25.280814 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:25.280850 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:25.280912 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:25.281259 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:25.281263 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:25.281266 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:14:25.281331 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:25.281333 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:25.282395 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:25.282396 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:25.283537 (+ 1141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:25.283539 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:25.284368 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:25.284369 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:25.284443 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:25.284455 (+ 12us) client_negotiation.cc:769] Sending connection context
0923 23:14:25.284549 (+ 94us) client_negotiation.cc:240] Negotiation successful
0923 23:14:25.284601 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":71,"threads_started":1}
I20260923 23:14:25.284698 27099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:25.280896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35486 (local address 127.24.227.194:44665)
0923 23:14:25.281063 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:25.281069 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:25.281081 (+ 12us) server_negotiation.cc:407] Connection header received
0923 23:14:25.281120 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:25.281122 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:25.281138 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:25.281406 (+ 268us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:14:25.281467 (+ 61us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:25.282320 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:25.283626 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:25.284281 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:25.284361 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:25.284415 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:25.284567 (+ 152us) server_negotiation.cc:299] Negotiation successful
0923 23:14:25.284602 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":107,"threads_started":1}
I20260923 23:14:25.285019 27097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:25.280675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.195:39787 (local address 127.0.0.1:33452)
0923 23:14:25.281252 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:25.281262 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:25.281307 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:25.281490 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:25.281490 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:25.281492 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:14:25.281530 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:25.281531 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:25.282743 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:25.282744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:25.283717 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:25.283718 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:25.284705 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:25.284706 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:25.284846 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:25.284852 (+ 6us) client_negotiation.cc:769] Sending connection context
0923 23:14:25.284901 (+ 49us) client_negotiation.cc:240] Negotiation successful
0923 23:14:25.284937 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":564,"mutex_wait_us":1,"thread_start_us":48,"threads_started":1}
I20260923 23:14:25.285324 27101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:25.281100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33452 (local address 127.24.227.195:39787)
0923 23:14:25.281250 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:25.281253 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:25.281361 (+ 108us) server_negotiation.cc:407] Connection header received
0923 23:14:25.281401 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:25.281403 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:25.281418 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:25.281493 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:14:25.281605 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:25.282659 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:25.283799 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:25.284551 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:25.284624 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:25.284671 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:25.285185 (+ 514us) server_negotiation.cc:299] Negotiation successful
0923 23:14:25.285226 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":74,"threads_started":1}
I20260923 23:14:25.285398 26876 tablet_service.cc:1467] Processing CreateTablet for tablet 8e54924971fe4565a7274a6716b84b42 (DEFAULT_TABLE table=test-table [id=3bdca4bac4264e9fa48cdf3f4715644e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 23:14:25.285604 27098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:25.280680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.193:36649 (local address 127.0.0.1:60900)
0923 23:14:25.281572 (+ 892us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:25.281582 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:25.281633 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:25.281945 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:25.281946 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:25.281948 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:14:25.281989 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:25.281990 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:25.283007 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:25.283008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:25.283989 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:25.283992 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:25.285375 (+ 1383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:25.285377 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:25.285436 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:25.285439 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 23:14:25.285502 (+ 63us) client_negotiation.cc:240] Negotiation successful
0923 23:14:25.285538 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":880,"thread_start_us":70,"threads_started":1}
I20260923 23:14:25.285765 26876 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e54924971fe4565a7274a6716b84b42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:14:25.285835 27100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:25.280921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60900 (local address 127.24.227.193:36649)
0923 23:14:25.281080 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:25.281084 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:25.281631 (+ 547us) server_negotiation.cc:407] Connection header received
0923 23:14:25.281835 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:25.281838 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:25.281854 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:25.281934 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:14:25.282047 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:25.282935 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:25.284072 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:25.285036 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:25.285271 (+ 235us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:25.285330 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:25.285692 (+ 362us) server_negotiation.cc:299] Negotiation successful
0923 23:14:25.285739 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":116,"threads_started":1}
I20260923 23:14:25.285914 27016 tablet_service.cc:1467] Processing CreateTablet for tablet 8e54924971fe4565a7274a6716b84b42 (DEFAULT_TABLE table=test-table [id=3bdca4bac4264e9fa48cdf3f4715644e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 23:14:25.286209 27016 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e54924971fe4565a7274a6716b84b42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:14:25.286401 26736 tablet_service.cc:1467] Processing CreateTablet for tablet 8e54924971fe4565a7274a6716b84b42 (DEFAULT_TABLE table=test-table [id=3bdca4bac4264e9fa48cdf3f4715644e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 23:14:25.286679 26736 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e54924971fe4565a7274a6716b84b42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:14:25.288605 27102 tablet_bootstrap.cc:492] T 8e54924971fe4565a7274a6716b84b42 P 08c54376cc9d473596a9b8c014946779: Bootstrap starting.
I20260923 23:14:25.289214 27102 tablet_bootstrap.cc:654] T 8e54924971fe4565a7274a6716b84b42 P 08c54376cc9d473596a9b8c014946779: Neither blocks nor log segments found. Creating new log.
I20260923 23:14:25.289407 27103 tablet_bootstrap.cc:492] T 8e54924971fe4565a7274a6716b84b42 P 643a0fc32e094e1eb52717bd61a2fd85: Bootstrap starting.
I20260923 23:14:25.289556 27102 log.cc:824] T 8e54924971fe4565a7274a6716b84b42 P 08c54376cc9d473596a9b8c014946779: Log is configured to *not* fsync() on all Append() calls
I20260923 23:14:25.290212 27103 tablet_bootstrap.cc:654] T 8e54924971fe4565a7274a6716b84b42 P 643a0fc32e094e1eb52717bd61a2fd85: Neither blocks nor log segments found. Creating new log.
I20260923 23:14:25.290354 27102 tablet_bootstrap.cc:492] T 8e54924971fe4565a7274a6716b84b42 P 08c54376cc9d473596a9b8c014946779: No bootstrap required, opened a new log
I20260923 23:14:25.290424 27102 ts_tablet_manager.cc:1397] T 8e54924971fe4565a7274a6716b84b42 P 08c54376cc9d473596a9b8c014946779: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 23:14:25.290385 27105 tablet_bootstrap.cc:492] T 8e54924971fe4565a7274a6716b84b42 P b92ee785a3c04711abeb66c9af684149: Bootstrap starting.
I20260923 23:14:25.290525 27103 log.cc:824] T 8e54924971fe4565a7274a6716b84b42 P 643a0fc32e094e1eb52717bd61a2fd85: Log is configured to *not* fsync() on all Append() calls
I20260923 23:14:25.291102 27105 tablet_bootstrap.cc:654] T 8e54924971fe4565a7274a6716b84b42 P b92ee785a3c04711abeb66c9af684149: Neither blocks nor log segments found. Creating new log.
I20260923 23:14:25.291268 27103 tablet_bootstrap.cc:492] T 8e54924971fe4565a7274a6716b84b42 P 643a0fc32e094e1eb52717bd61a2fd85: No bootstrap required, opened a new log
I20260923 23:14:25.291334 27103 ts_tablet_manager.cc:1397] T 8e54924971fe4565a7274a6716b84b42 P 643a0fc32e094e1eb52717bd61a2fd85: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 23:14:25.291443 27105 log.cc:824] T 8e54924971fe4565a7274a6716b84b42 P b92ee785a3c04711abeb66c9af684149: Log is configured to *not* fsync() on all Append() calls
I20260923 23:14:25.292100 27105 tablet_bootstrap.cc:492] T 8e54924971fe4565a7274a6716b84b42 P b92ee785a3c04711abeb66c9af684149: No bootstrap required, opened a new log
I20260923 23:14:25.292183 27105 ts_tablet_manager.cc:1397] T 8e54924971fe4565a7274a6716b84b42 P b92ee785a3c04711abeb66c9af684149: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 23:14:25.292409 27102 raft_consensus.cc:348] T 8e54924971fe4565a7274a6716b84b42 P 08c54376cc9d473596a9b8c014946779 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643a0fc32e094e1eb52717bd61a2fd85" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39787 } } peers { permanent_uuid: "08c54376cc9d473596a9b8c014946779" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 44665 } } peers { permanent_uuid: "b92ee785a3c04711abeb66c9af684149" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 36649 } }
I20260923 23:14:25.292582 27102 raft_consensus.cc:374] T 8e54924971fe4565a7274a6716b84b42 P 08c54376cc9d473596a9b8c014946779 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:14:25.292615 27102 raft_consensus.cc:729] T 8e54924971fe4565a7274a6716b84b42 P 08c54376cc9d473596a9b8c014946779 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08c54376cc9d473596a9b8c014946779, State: Initialized, Role: FOLLOWER
I20260923 23:14:25.292747 27102 consensus_queue.cc:260] T 8e54924971fe4565a7274a6716b84b42 P 08c54376cc9d473596a9b8c014946779 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643a0fc32e094e1eb52717bd61a2fd85" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39787 } } peers { permanent_uuid: "08c54376cc9d473596a9b8c014946779" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 44665 } } peers { permanent_uuid: "b92ee785a3c04711abeb66c9af684149" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 36649 } }
I20260923 23:14:25.292999 27102 ts_tablet_manager.cc:1428] T 8e54924971fe4565a7274a6716b84b42 P 08c54376cc9d473596a9b8c014946779: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 23:14:25.293025 26942 heartbeater.cc:503] Master 127.24.227.254:41471 was elected leader, sending a full tablet report...
I20260923 23:14:25.293497 27103 raft_consensus.cc:348] T 8e54924971fe4565a7274a6716b84b42 P 643a0fc32e094e1eb52717bd61a2fd85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643a0fc32e094e1eb52717bd61a2fd85" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39787 } } peers { permanent_uuid: "08c54376cc9d473596a9b8c014946779" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 44665 } } peers { permanent_uuid: "b92ee785a3c04711abeb66c9af684149" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 36649 } }
I20260923 23:14:25.293639 27103 raft_consensus.cc:374] T 8e54924971fe4565a7274a6716b84b42 P 643a0fc32e094e1eb52717bd61a2fd85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:14:25.293679 27103 raft_consensus.cc:729] T 8e54924971fe4565a7274a6716b84b42 P 643a0fc32e094e1eb52717bd61a2fd85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 643a0fc32e094e1eb52717bd61a2fd85, State: Initialized, Role: FOLLOWER
I20260923 23:14:25.293803 27103 consensus_queue.cc:260] T 8e54924971fe4565a7274a6716b84b42 P 643a0fc32e094e1eb52717bd61a2fd85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643a0fc32e094e1eb52717bd61a2fd85" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39787 } } peers { permanent_uuid: "08c54376cc9d473596a9b8c014946779" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 44665 } } peers { permanent_uuid: "b92ee785a3c04711abeb66c9af684149" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 36649 } }
I20260923 23:14:25.294056 27103 ts_tablet_manager.cc:1428] T 8e54924971fe4565a7274a6716b84b42 P 643a0fc32e094e1eb52717bd61a2fd85: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 23:14:25.294175 27082 heartbeater.cc:503] Master 127.24.227.254:41471 was elected leader, sending a full tablet report...
I20260923 23:14:25.294181 27105 raft_consensus.cc:348] T 8e54924971fe4565a7274a6716b84b42 P b92ee785a3c04711abeb66c9af684149 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643a0fc32e094e1eb52717bd61a2fd85" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39787 } } peers { permanent_uuid: "08c54376cc9d473596a9b8c014946779" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 44665 } } peers { permanent_uuid: "b92ee785a3c04711abeb66c9af684149" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 36649 } }
I20260923 23:14:25.294327 27105 raft_consensus.cc:374] T 8e54924971fe4565a7274a6716b84b42 P b92ee785a3c04711abeb66c9af684149 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:14:25.294389 27105 raft_consensus.cc:729] T 8e54924971fe4565a7274a6716b84b42 P b92ee785a3c04711abeb66c9af684149 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b92ee785a3c04711abeb66c9af684149, State: Initialized, Role: FOLLOWER
I20260923 23:14:25.294534 27105 consensus_queue.cc:260] T 8e54924971fe4565a7274a6716b84b42 P b92ee785a3c04711abeb66c9af684149 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643a0fc32e094e1eb52717bd61a2fd85" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39787 } } peers { permanent_uuid: "08c54376cc9d473596a9b8c014946779" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 44665 } } peers { permanent_uuid: "b92ee785a3c04711abeb66c9af684149" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 36649 } }
I20260923 23:14:25.294773 27105 ts_tablet_manager.cc:1428] T 8e54924971fe4565a7274a6716b84b42 P b92ee785a3c04711abeb66c9af684149: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 23:14:25.294837 26802 heartbeater.cc:503] Master 127.24.227.254:41471 was elected leader, sending a full tablet report...
W20260923 23:14:25.353292 26943 tablet.cc:2369] T 8e54924971fe4565a7274a6716b84b42 P 08c54376cc9d473596a9b8c014946779: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 23:14:25.459118 26803 tablet.cc:2369] T 8e54924971fe4565a7274a6716b84b42 P b92ee785a3c04711abeb66c9af684149: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 23:14:25.496016 27083 tablet.cc:2369] T 8e54924971fe4565a7274a6716b84b42 P 643a0fc32e094e1eb52717bd61a2fd85: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 23:14:25.594125 27108 raft_consensus.cc:482] T 8e54924971fe4565a7274a6716b84b42 P 08c54376cc9d473596a9b8c014946779 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 23:14:25.594277 27108 raft_consensus.cc:504] T 8e54924971fe4565a7274a6716b84b42 P 08c54376cc9d473596a9b8c014946779 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643a0fc32e094e1eb52717bd61a2fd85" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39787 } } peers { permanent_uuid: "08c54376cc9d473596a9b8c014946779" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 44665 } } peers { permanent_uuid: "b92ee785a3c04711abeb66c9af684149" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 36649 } }
I20260923 23:14:25.594655 27108 leader_election.cc:288] T 8e54924971fe4565a7274a6716b84b42 P 08c54376cc9d473596a9b8c014946779 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 643a0fc32e094e1eb52717bd61a2fd85 (127.24.227.195:39787), b92ee785a3c04711abeb66c9af684149 (127.24.227.193:36649)
I20260923 23:14:25.597397 27100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:25.595008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.194:32891 (local address 127.24.227.193:36649)
0923 23:14:25.595059 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:25.595061 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:25.595069 (+ 8us) server_negotiation.cc:407] Connection header received
0923 23:14:25.595101 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:25.595102 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:25.595108 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:25.595172 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:14:25.595324 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:25.595786 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:25.596721 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:25.597062 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:25.597144 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:25.597152 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:25.597301 (+ 149us) server_negotiation.cc:299] Negotiation successful
0923 23:14:25.597333 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260923 23:14:25.597419 26835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:25.594848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.193:36649 (local address 127.24.227.194:32891)
0923 23:14:25.594917 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:25.594951 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:25.594998 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:25.595176 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:25.595177 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:25.595179 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:14:25.595252 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:25.595254 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:25.595887 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:25.595888 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:25.596624 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:25.596626 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:25.597140 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:25.597141 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:25.597226 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:25.597229 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 23:14:25.597294 (+ 65us) client_negotiation.cc:240] Negotiation successful
0923 23:14:25.597328 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260923 23:14:25.597455 27101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:25.594918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.194:40503 (local address 127.24.227.195:39787)
0923 23:14:25.594988 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:25.594990 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:25.595077 (+ 87us) server_negotiation.cc:407] Connection header received
0923 23:14:25.595140 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:25.595141 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:25.595151 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:25.595218 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:14:25.595369 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:25.595830 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:25.596754 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:25.597104 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:25.597183 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:25.597190 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:25.597347 (+ 157us) server_negotiation.cc:299] Negotiation successful
0923 23:14:25.597384 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260923 23:14:25.597723 27111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:25.594866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.195:39787 (local address 127.24.227.194:40503)
0923 23:14:25.594990 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:25.595001 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:25.595083 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:25.595233 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:25.595234 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:25.595236 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:14:25.595303 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:25.595305 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:25.595965 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:25.595966 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:25.596651 (+ 685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:25.596653 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:25.597193 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:25.597195 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:25.597265 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:25.597268 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 23:14:25.597339 (+ 71us) client_negotiation.cc:240] Negotiation successful
0923 23:14:25.597376 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":64,"threads_started":1}
I20260923 23:14:25.597815 26756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e54924971fe4565a7274a6716b84b42" candidate_uuid: "08c54376cc9d473596a9b8c014946779" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b92ee785a3c04711abeb66c9af684149" is_pre_election: true
I20260923 23:14:25.597963 26756 raft_consensus.cc:2454] T 8e54924971fe4565a7274a6716b84b42 P b92ee785a3c04711abeb66c9af684149 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 08c54376cc9d473596a9b8c014946779 in term 0.
I20260923 23:14:25.598031 27036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e54924971fe4565a7274a6716b84b42" candidate_uuid: "08c54376cc9d473596a9b8c014946779" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643a0fc32e094e1eb52717bd61a2fd85" is_pre_election: true
I20260923 23:14:25.598214 27036 raft_consensus.cc:2454] T 8e54924971fe4565a7274a6716b84b42 P 643a0fc32e094e1eb52717bd61a2fd85 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 08c54376cc9d473596a9b8c014946779 in term 0.
I20260923 23:14:25.598263 26830 leader_election.cc:302] T 8e54924971fe4565a7274a6716b84b42 P 08c54376cc9d473596a9b8c014946779 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08c54376cc9d473596a9b8c014946779, b92ee785a3c04711abeb66c9af684149; no voters:
I20260923 23:14:25.598380 27108 raft_consensus.cc:2781] T 8e54924971fe4565a7274a6716b84b42 P 08c54376cc9d473596a9b8c014946779 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 23:14:25.598454 27108 raft_consensus.cc:482] T 8e54924971fe4565a7274a6716b84b42 P 08c54376cc9d473596a9b8c014946779 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 23:14:25.598491 27108 raft_consensus.cc:3037] T 8e54924971fe4565a7274a6716b84b42 P 08c54376cc9d473596a9b8c014946779 [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:14:25.599236 27108 raft_consensus.cc:504] T 8e54924971fe4565a7274a6716b84b42 P 08c54376cc9d473596a9b8c014946779 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643a0fc32e094e1eb52717bd61a2fd85" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39787 } } peers { permanent_uuid: "08c54376cc9d473596a9b8c014946779" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 44665 } } peers { permanent_uuid: "b92ee785a3c04711abeb66c9af684149" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 36649 } }
I20260923 23:14:25.599371 27108 leader_election.cc:288] T 8e54924971fe4565a7274a6716b84b42 P 08c54376cc9d473596a9b8c014946779 [CANDIDATE]: Term 1 election: Requested vote from peers 643a0fc32e094e1eb52717bd61a2fd85 (127.24.227.195:39787), b92ee785a3c04711abeb66c9af684149 (127.24.227.193:36649)
I20260923 23:14:25.599507 26756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e54924971fe4565a7274a6716b84b42" candidate_uuid: "08c54376cc9d473596a9b8c014946779" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b92ee785a3c04711abeb66c9af684149"
I20260923 23:14:25.599507 27036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e54924971fe4565a7274a6716b84b42" candidate_uuid: "08c54376cc9d473596a9b8c014946779" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643a0fc32e094e1eb52717bd61a2fd85"
I20260923 23:14:25.599589 26756 raft_consensus.cc:3037] T 8e54924971fe4565a7274a6716b84b42 P b92ee785a3c04711abeb66c9af684149 [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:14:25.599612 27036 raft_consensus.cc:3037] T 8e54924971fe4565a7274a6716b84b42 P 643a0fc32e094e1eb52717bd61a2fd85 [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:14:25.600277 26756 raft_consensus.cc:2454] T 8e54924971fe4565a7274a6716b84b42 P b92ee785a3c04711abeb66c9af684149 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 08c54376cc9d473596a9b8c014946779 in term 1.
I20260923 23:14:25.600407 27036 raft_consensus.cc:2454] T 8e54924971fe4565a7274a6716b84b42 P 643a0fc32e094e1eb52717bd61a2fd85 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 08c54376cc9d473596a9b8c014946779 in term 1.
I20260923 23:14:25.600445 26830 leader_election.cc:302] T 8e54924971fe4565a7274a6716b84b42 P 08c54376cc9d473596a9b8c014946779 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08c54376cc9d473596a9b8c014946779, b92ee785a3c04711abeb66c9af684149; no voters:
I20260923 23:14:25.600543 27108 raft_consensus.cc:2781] T 8e54924971fe4565a7274a6716b84b42 P 08c54376cc9d473596a9b8c014946779 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 23:14:25.600739 27108 raft_consensus.cc:686] T 8e54924971fe4565a7274a6716b84b42 P 08c54376cc9d473596a9b8c014946779 [term 1 LEADER]: Becoming Leader. State: Replica: 08c54376cc9d473596a9b8c014946779, State: Running, Role: LEADER
I20260923 23:14:25.600833 27108 consensus_queue.cc:237] T 8e54924971fe4565a7274a6716b84b42 P 08c54376cc9d473596a9b8c014946779 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643a0fc32e094e1eb52717bd61a2fd85" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39787 } } peers { permanent_uuid: "08c54376cc9d473596a9b8c014946779" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 44665 } } peers { permanent_uuid: "b92ee785a3c04711abeb66c9af684149" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 36649 } }
I20260923 23:14:25.601692 26599 catalog_manager.cc:5495] T 8e54924971fe4565a7274a6716b84b42 P 08c54376cc9d473596a9b8c014946779 reported cstate change: term changed from 0 to 1, leader changed from <none> to 08c54376cc9d473596a9b8c014946779 (127.24.227.194). New cstate: current_term: 1 leader_uuid: "08c54376cc9d473596a9b8c014946779" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643a0fc32e094e1eb52717bd61a2fd85" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39787 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "08c54376cc9d473596a9b8c014946779" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 44665 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b92ee785a3c04711abeb66c9af684149" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 36649 } health_report { overall_health: UNKNOWN } } }
I20260923 23:14:25.639741 27099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:25.637550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35488 (local address 127.24.227.194:44665)
0923 23:14:25.637614 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:25.637616 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:25.637630 (+ 14us) server_negotiation.cc:407] Connection header received
0923 23:14:25.637680 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:25.637680 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:25.637693 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:25.637747 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 23:14:25.637869 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:25.638341 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:25.639095 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:25.639267 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:25.639314 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:25.639478 (+ 164us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 23:14:25.639573 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 23:14:25.639628 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:25.639674 (+ 46us) server_negotiation.cc:299] Negotiation successful
0923 23:14:25.639691 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260923 23:14:25.641288 27036 raft_consensus.cc:1261] T 8e54924971fe4565a7274a6716b84b42 P 643a0fc32e094e1eb52717bd61a2fd85 [term 1 FOLLOWER]: Refusing update from remote peer 08c54376cc9d473596a9b8c014946779: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 23:14:25.641566 26756 raft_consensus.cc:1261] T 8e54924971fe4565a7274a6716b84b42 P b92ee785a3c04711abeb66c9af684149 [term 1 FOLLOWER]: Refusing update from remote peer 08c54376cc9d473596a9b8c014946779: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 23:14:25.641639 27108 consensus_queue.cc:1035] T 8e54924971fe4565a7274a6716b84b42 P 08c54376cc9d473596a9b8c014946779 [LEADER]: Connected to new peer: Peer: permanent_uuid: "643a0fc32e094e1eb52717bd61a2fd85" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 23:14:25.641819 27108 consensus_queue.cc:1035] T 8e54924971fe4565a7274a6716b84b42 P 08c54376cc9d473596a9b8c014946779 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b92ee785a3c04711abeb66c9af684149" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 36649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 23:14:25.643853 27115 mvcc.cc:205] Tried to move back new op lower bound from 7332680768064307200 to 7332680767901659136. Current Snapshot: MvccSnapshot[applied={T|T < 7332680768064307200}]
I20260923 23:14:25.644675 27116 mvcc.cc:205] Tried to move back new op lower bound from 7332680768064307200 to 7332680767901659136. Current Snapshot: MvccSnapshot[applied={T|T < 7332680768064307200}]
I20260923 23:14:25.647449 26598 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52924:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 23:14:25.647540 26598 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52924:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 23:14:25.648263 26598 catalog_manager.cc:5782] T 00000000000000000000000000000000 P edb11fcb0fbd4223a79b05d5bfd4e8da: Sending DeleteTablet for 3 replicas of tablet 8e54924971fe4565a7274a6716b84b42
I20260923 23:14:25.648684 26736 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e54924971fe4565a7274a6716b84b42 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 23:14:25 UTC) from {username='kudu', principal='kudu/127.24.227.254@KRBTEST.COM'} at 127.0.0.1:60900
I20260923 23:14:25.648691 26876 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e54924971fe4565a7274a6716b84b42 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 23:14:25 UTC) from {username='kudu', principal='kudu/127.24.227.254@KRBTEST.COM'} at 127.0.0.1:35486
I20260923 23:14:25.648856 27125 tablet_replica.cc:330] T 8e54924971fe4565a7274a6716b84b42 P b92ee785a3c04711abeb66c9af684149: stopping tablet replica
I20260923 23:14:25.648870 27016 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e54924971fe4565a7274a6716b84b42 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 23:14:25 UTC) from {username='kudu', principal='kudu/127.24.227.254@KRBTEST.COM'} at 127.0.0.1:33452
I20260923 23:14:25.648949 27125 raft_consensus.cc:2229] T 8e54924971fe4565a7274a6716b84b42 P b92ee785a3c04711abeb66c9af684149 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 23:14:25.649046 27125 raft_consensus.cc:2258] T 8e54924971fe4565a7274a6716b84b42 P b92ee785a3c04711abeb66c9af684149 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 23:14:25.649268 27127 tablet_replica.cc:330] T 8e54924971fe4565a7274a6716b84b42 P 643a0fc32e094e1eb52717bd61a2fd85: stopping tablet replica
I20260923 23:14:25.649367 27127 raft_consensus.cc:2229] T 8e54924971fe4565a7274a6716b84b42 P 643a0fc32e094e1eb52717bd61a2fd85 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 23:14:25.649464 27127 raft_consensus.cc:2258] T 8e54924971fe4565a7274a6716b84b42 P 643a0fc32e094e1eb52717bd61a2fd85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 23:14:25.649495 26598 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52924:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 23:14:25.649639 26598 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 23:14:25.649653 27125 ts_tablet_manager.cc:1905] T 8e54924971fe4565a7274a6716b84b42 P b92ee785a3c04711abeb66c9af684149: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 23:14:25.650012 27127 ts_tablet_manager.cc:1905] T 8e54924971fe4565a7274a6716b84b42 P 643a0fc32e094e1eb52717bd61a2fd85: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 23:14:25.650609 27126 tablet_replica.cc:330] T 8e54924971fe4565a7274a6716b84b42 P 08c54376cc9d473596a9b8c014946779: stopping tablet replica
I20260923 23:14:25.650859 26736 tablet_service.cc:1467] Processing CreateTablet for tablet 1064065a85664dc3b7b5c0adada3015f (DEFAULT_TABLE table=test-table [id=a0b08e59c4ab42408c3115f35b5fc5a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 23:14:25.650959 26736 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1064065a85664dc3b7b5c0adada3015f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:14:25.651471 27127 ts_tablet_manager.cc:1918] T 8e54924971fe4565a7274a6716b84b42 P 643a0fc32e094e1eb52717bd61a2fd85: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 23:14:25.651523 27125 ts_tablet_manager.cc:1918] T 8e54924971fe4565a7274a6716b84b42 P b92ee785a3c04711abeb66c9af684149: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 23:14:25.651556 27127 log.cc:1196] T 8e54924971fe4565a7274a6716b84b42 P 643a0fc32e094e1eb52717bd61a2fd85: Deleting WAL directory at /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/minicluster-data/ts-2/wal/wals/8e54924971fe4565a7274a6716b84b42
I20260923 23:14:25.651583 27125 log.cc:1196] T 8e54924971fe4565a7274a6716b84b42 P b92ee785a3c04711abeb66c9af684149: Deleting WAL directory at /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/minicluster-data/ts-0/wal/wals/8e54924971fe4565a7274a6716b84b42
I20260923 23:14:25.651815 27127 ts_tablet_manager.cc:1939] T 8e54924971fe4565a7274a6716b84b42 P 643a0fc32e094e1eb52717bd61a2fd85: Deleting consensus metadata
I20260923 23:14:25.651829 27125 ts_tablet_manager.cc:1939] T 8e54924971fe4565a7274a6716b84b42 P b92ee785a3c04711abeb66c9af684149: Deleting consensus metadata
I20260923 23:14:25.652236 26586 catalog_manager.cc:4841] TS b92ee785a3c04711abeb66c9af684149 (127.24.227.193:36649): tablet 8e54924971fe4565a7274a6716b84b42 (table test-table [id=3bdca4bac4264e9fa48cdf3f4715644e]) successfully deleted
I20260923 23:14:25.652297 27105 tablet_bootstrap.cc:492] T 1064065a85664dc3b7b5c0adada3015f P b92ee785a3c04711abeb66c9af684149: Bootstrap starting.
I20260923 23:14:25.652369 27016 tablet_service.cc:1467] Processing CreateTablet for tablet 1064065a85664dc3b7b5c0adada3015f (DEFAULT_TABLE table=test-table [id=a0b08e59c4ab42408c3115f35b5fc5a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 23:14:25.652494 27016 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1064065a85664dc3b7b5c0adada3015f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:14:25.652668 27105 tablet_bootstrap.cc:654] T 1064065a85664dc3b7b5c0adada3015f P b92ee785a3c04711abeb66c9af684149: Neither blocks nor log segments found. Creating new log.
I20260923 23:14:25.653219 27105 tablet_bootstrap.cc:492] T 1064065a85664dc3b7b5c0adada3015f P b92ee785a3c04711abeb66c9af684149: No bootstrap required, opened a new log
I20260923 23:14:25.653273 27105 ts_tablet_manager.cc:1397] T 1064065a85664dc3b7b5c0adada3015f P b92ee785a3c04711abeb66c9af684149: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260923 23:14:25.653409 27105 raft_consensus.cc:348] T 1064065a85664dc3b7b5c0adada3015f P b92ee785a3c04711abeb66c9af684149 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b92ee785a3c04711abeb66c9af684149" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 36649 } } peers { permanent_uuid: "08c54376cc9d473596a9b8c014946779" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 44665 } } peers { permanent_uuid: "643a0fc32e094e1eb52717bd61a2fd85" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39787 } }
I20260923 23:14:25.653472 27105 raft_consensus.cc:374] T 1064065a85664dc3b7b5c0adada3015f P b92ee785a3c04711abeb66c9af684149 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:14:25.653498 27105 raft_consensus.cc:729] T 1064065a85664dc3b7b5c0adada3015f P b92ee785a3c04711abeb66c9af684149 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b92ee785a3c04711abeb66c9af684149, State: Initialized, Role: FOLLOWER
I20260923 23:14:25.653514 27103 tablet_bootstrap.cc:492] T 1064065a85664dc3b7b5c0adada3015f P 643a0fc32e094e1eb52717bd61a2fd85: Bootstrap starting.
I20260923 23:14:25.653537 27105 consensus_queue.cc:260] T 1064065a85664dc3b7b5c0adada3015f P b92ee785a3c04711abeb66c9af684149 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b92ee785a3c04711abeb66c9af684149" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 36649 } } peers { permanent_uuid: "08c54376cc9d473596a9b8c014946779" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 44665 } } peers { permanent_uuid: "643a0fc32e094e1eb52717bd61a2fd85" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39787 } }
I20260923 23:14:25.653635 27105 ts_tablet_manager.cc:1428] T 1064065a85664dc3b7b5c0adada3015f P b92ee785a3c04711abeb66c9af684149: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:25.653611 26585 catalog_manager.cc:4841] TS 643a0fc32e094e1eb52717bd61a2fd85 (127.24.227.195:39787): tablet 8e54924971fe4565a7274a6716b84b42 (table test-table [id=3bdca4bac4264e9fa48cdf3f4715644e]) successfully deleted
I20260923 23:14:25.653885 27103 tablet_bootstrap.cc:654] T 1064065a85664dc3b7b5c0adada3015f P 643a0fc32e094e1eb52717bd61a2fd85: Neither blocks nor log segments found. Creating new log.
I20260923 23:14:25.653941 26876 tablet_service.cc:1467] Processing CreateTablet for tablet 1064065a85664dc3b7b5c0adada3015f (DEFAULT_TABLE table=test-table [id=a0b08e59c4ab42408c3115f35b5fc5a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 23:14:25.654330 27126 raft_consensus.cc:2229] T 8e54924971fe4565a7274a6716b84b42 P 08c54376cc9d473596a9b8c014946779 [term 1 LEADER]: Raft consensus shutting down.
I20260923 23:14:25.654353 26876 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1064065a85664dc3b7b5c0adada3015f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:14:25.654421 27126 raft_consensus.cc:2258] T 8e54924971fe4565a7274a6716b84b42 P 08c54376cc9d473596a9b8c014946779 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 23:14:25.654528 27103 tablet_bootstrap.cc:492] T 1064065a85664dc3b7b5c0adada3015f P 643a0fc32e094e1eb52717bd61a2fd85: No bootstrap required, opened a new log
I20260923 23:14:25.654598 27103 ts_tablet_manager.cc:1397] T 1064065a85664dc3b7b5c0adada3015f P 643a0fc32e094e1eb52717bd61a2fd85: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260923 23:14:25.654737 27103 raft_consensus.cc:348] T 1064065a85664dc3b7b5c0adada3015f P 643a0fc32e094e1eb52717bd61a2fd85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b92ee785a3c04711abeb66c9af684149" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 36649 } } peers { permanent_uuid: "08c54376cc9d473596a9b8c014946779" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 44665 } } peers { permanent_uuid: "643a0fc32e094e1eb52717bd61a2fd85" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39787 } }
I20260923 23:14:25.654803 27103 raft_consensus.cc:374] T 1064065a85664dc3b7b5c0adada3015f P 643a0fc32e094e1eb52717bd61a2fd85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:14:25.654827 27103 raft_consensus.cc:729] T 1064065a85664dc3b7b5c0adada3015f P 643a0fc32e094e1eb52717bd61a2fd85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 643a0fc32e094e1eb52717bd61a2fd85, State: Initialized, Role: FOLLOWER
I20260923 23:14:25.654870 27103 consensus_queue.cc:260] T 1064065a85664dc3b7b5c0adada3015f P 643a0fc32e094e1eb52717bd61a2fd85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b92ee785a3c04711abeb66c9af684149" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 36649 } } peers { permanent_uuid: "08c54376cc9d473596a9b8c014946779" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 44665 } } peers { permanent_uuid: "643a0fc32e094e1eb52717bd61a2fd85" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39787 } }
I20260923 23:14:25.654960 27103 ts_tablet_manager.cc:1428] T 1064065a85664dc3b7b5c0adada3015f P 643a0fc32e094e1eb52717bd61a2fd85: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:25.655243 27126 ts_tablet_manager.cc:1905] T 8e54924971fe4565a7274a6716b84b42 P 08c54376cc9d473596a9b8c014946779: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 23:14:25.655608 27102 tablet_bootstrap.cc:492] T 1064065a85664dc3b7b5c0adada3015f P 08c54376cc9d473596a9b8c014946779: Bootstrap starting.
I20260923 23:14:25.656023 27102 tablet_bootstrap.cc:654] T 1064065a85664dc3b7b5c0adada3015f P 08c54376cc9d473596a9b8c014946779: Neither blocks nor log segments found. Creating new log.
I20260923 23:14:25.656611 27102 tablet_bootstrap.cc:492] T 1064065a85664dc3b7b5c0adada3015f P 08c54376cc9d473596a9b8c014946779: No bootstrap required, opened a new log
I20260923 23:14:25.656667 27102 ts_tablet_manager.cc:1397] T 1064065a85664dc3b7b5c0adada3015f P 08c54376cc9d473596a9b8c014946779: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260923 23:14:25.656800 27102 raft_consensus.cc:348] T 1064065a85664dc3b7b5c0adada3015f P 08c54376cc9d473596a9b8c014946779 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b92ee785a3c04711abeb66c9af684149" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 36649 } } peers { permanent_uuid: "08c54376cc9d473596a9b8c014946779" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 44665 } } peers { permanent_uuid: "643a0fc32e094e1eb52717bd61a2fd85" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39787 } }
I20260923 23:14:25.656865 27102 raft_consensus.cc:374] T 1064065a85664dc3b7b5c0adada3015f P 08c54376cc9d473596a9b8c014946779 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:14:25.656891 27102 raft_consensus.cc:729] T 1064065a85664dc3b7b5c0adada3015f P 08c54376cc9d473596a9b8c014946779 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08c54376cc9d473596a9b8c014946779, State: Initialized, Role: FOLLOWER
I20260923 23:14:25.656931 27102 consensus_queue.cc:260] T 1064065a85664dc3b7b5c0adada3015f P 08c54376cc9d473596a9b8c014946779 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b92ee785a3c04711abeb66c9af684149" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 36649 } } peers { permanent_uuid: "08c54376cc9d473596a9b8c014946779" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 44665 } } peers { permanent_uuid: "643a0fc32e094e1eb52717bd61a2fd85" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39787 } }
I20260923 23:14:25.657035 27102 ts_tablet_manager.cc:1428] T 1064065a85664dc3b7b5c0adada3015f P 08c54376cc9d473596a9b8c014946779: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:25.658178 27126 ts_tablet_manager.cc:1918] T 8e54924971fe4565a7274a6716b84b42 P 08c54376cc9d473596a9b8c014946779: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 23:14:25.658247 27126 log.cc:1196] T 8e54924971fe4565a7274a6716b84b42 P 08c54376cc9d473596a9b8c014946779: Deleting WAL directory at /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/minicluster-data/ts-1/wal/wals/8e54924971fe4565a7274a6716b84b42
I20260923 23:14:25.658468 27126 ts_tablet_manager.cc:1939] T 8e54924971fe4565a7274a6716b84b42 P 08c54376cc9d473596a9b8c014946779: Deleting consensus metadata
I20260923 23:14:25.658972 26583 catalog_manager.cc:4841] TS 08c54376cc9d473596a9b8c014946779 (127.24.227.194:44665): tablet 8e54924971fe4565a7274a6716b84b42 (table test-table [id=3bdca4bac4264e9fa48cdf3f4715644e]) successfully deleted
I20260923 23:14:25.709609 27110 raft_consensus.cc:482] T 1064065a85664dc3b7b5c0adada3015f P b92ee785a3c04711abeb66c9af684149 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 23:14:25.709697 27110 raft_consensus.cc:504] T 1064065a85664dc3b7b5c0adada3015f P b92ee785a3c04711abeb66c9af684149 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b92ee785a3c04711abeb66c9af684149" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 36649 } } peers { permanent_uuid: "08c54376cc9d473596a9b8c014946779" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 44665 } } peers { permanent_uuid: "643a0fc32e094e1eb52717bd61a2fd85" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39787 } }
I20260923 23:14:25.710021 27110 leader_election.cc:288] T 1064065a85664dc3b7b5c0adada3015f P b92ee785a3c04711abeb66c9af684149 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 08c54376cc9d473596a9b8c014946779 (127.24.227.194:44665), 643a0fc32e094e1eb52717bd61a2fd85 (127.24.227.195:39787)
I20260923 23:14:25.713243 27101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:25.710332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.193:51595 (local address 127.24.227.195:39787)
0923 23:14:25.710402 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:25.710404 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:25.710463 (+ 59us) server_negotiation.cc:407] Connection header received
0923 23:14:25.710498 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:25.710499 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:25.710509 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:25.710569 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:14:25.710748 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:25.711197 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:25.712519 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:25.712908 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:25.712983 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:25.712991 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:25.713149 (+ 158us) server_negotiation.cc:299] Negotiation successful
0923 23:14:25.713189 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260923 23:14:25.713253 27132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:25.710177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.195:39787 (local address 127.24.227.193:51595)
0923 23:14:25.710381 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:25.710392 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:25.710453 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:25.710585 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:25.710586 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:25.710588 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:14:25.710660 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:25.710661 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:25.711269 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:25.711270 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:25.712428 (+ 1158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:25.712431 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:25.713008 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:25.713009 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:25.713069 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:25.713072 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 23:14:25.713141 (+ 69us) client_negotiation.cc:240] Negotiation successful
0923 23:14:25.713175 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":75,"threads_started":1}
I20260923 23:14:25.713555 27099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:25.710331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.193:57485 (local address 127.24.227.194:44665)
0923 23:14:25.710374 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:25.710376 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:25.710604 (+ 228us) server_negotiation.cc:407] Connection header received
0923 23:14:25.710701 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:25.710701 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:25.710710 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:25.710748 (+ 38us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:14:25.711405 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:25.711804 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:25.712657 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:25.713157 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:25.713224 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:25.713230 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:25.713462 (+ 232us) server_negotiation.cc:299] Negotiation successful
0923 23:14:25.713504 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 23:14:25.713536 27131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:25.710181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.194:44665 (local address 127.24.227.193:57485)
0923 23:14:25.710524 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:25.710534 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:25.710639 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:25.711271 (+ 632us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:25.711271 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:25.711272 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:14:25.711327 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:25.711328 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:25.711897 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:25.711898 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:25.712574 (+ 676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:25.712576 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:25.713264 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:25.713265 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:25.713338 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:25.713341 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 23:14:25.713428 (+ 87us) client_negotiation.cc:240] Negotiation successful
0923 23:14:25.713463 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":71,"threads_started":1}
I20260923 23:14:25.713776 27036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1064065a85664dc3b7b5c0adada3015f" candidate_uuid: "b92ee785a3c04711abeb66c9af684149" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643a0fc32e094e1eb52717bd61a2fd85" is_pre_election: true
I20260923 23:14:25.713853 27036 raft_consensus.cc:2454] T 1064065a85664dc3b7b5c0adada3015f P 643a0fc32e094e1eb52717bd61a2fd85 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b92ee785a3c04711abeb66c9af684149 in term 0.
I20260923 23:14:25.714062 26689 leader_election.cc:302] T 1064065a85664dc3b7b5c0adada3015f P b92ee785a3c04711abeb66c9af684149 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 643a0fc32e094e1eb52717bd61a2fd85, b92ee785a3c04711abeb66c9af684149; no voters:
I20260923 23:14:25.714042 26896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1064065a85664dc3b7b5c0adada3015f" candidate_uuid: "b92ee785a3c04711abeb66c9af684149" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08c54376cc9d473596a9b8c014946779" is_pre_election: true
I20260923 23:14:25.714170 26896 raft_consensus.cc:2454] T 1064065a85664dc3b7b5c0adada3015f P 08c54376cc9d473596a9b8c014946779 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b92ee785a3c04711abeb66c9af684149 in term 0.
I20260923 23:14:25.714218 27110 raft_consensus.cc:2781] T 1064065a85664dc3b7b5c0adada3015f P b92ee785a3c04711abeb66c9af684149 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 23:14:25.714259 27110 raft_consensus.cc:482] T 1064065a85664dc3b7b5c0adada3015f P b92ee785a3c04711abeb66c9af684149 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 23:14:25.714287 27110 raft_consensus.cc:3037] T 1064065a85664dc3b7b5c0adada3015f P b92ee785a3c04711abeb66c9af684149 [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:14:25.714928 27110 raft_consensus.cc:504] T 1064065a85664dc3b7b5c0adada3015f P b92ee785a3c04711abeb66c9af684149 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b92ee785a3c04711abeb66c9af684149" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 36649 } } peers { permanent_uuid: "08c54376cc9d473596a9b8c014946779" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 44665 } } peers { permanent_uuid: "643a0fc32e094e1eb52717bd61a2fd85" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39787 } }
I20260923 23:14:25.715055 27110 leader_election.cc:288] T 1064065a85664dc3b7b5c0adada3015f P b92ee785a3c04711abeb66c9af684149 [CANDIDATE]: Term 1 election: Requested vote from peers 08c54376cc9d473596a9b8c014946779 (127.24.227.194:44665), 643a0fc32e094e1eb52717bd61a2fd85 (127.24.227.195:39787)
I20260923 23:14:25.715175 26896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1064065a85664dc3b7b5c0adada3015f" candidate_uuid: "b92ee785a3c04711abeb66c9af684149" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08c54376cc9d473596a9b8c014946779"
I20260923 23:14:25.715236 26896 raft_consensus.cc:3037] T 1064065a85664dc3b7b5c0adada3015f P 08c54376cc9d473596a9b8c014946779 [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:14:25.715778 26896 raft_consensus.cc:2454] T 1064065a85664dc3b7b5c0adada3015f P 08c54376cc9d473596a9b8c014946779 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b92ee785a3c04711abeb66c9af684149 in term 1.
I20260923 23:14:25.715911 27036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1064065a85664dc3b7b5c0adada3015f" candidate_uuid: "b92ee785a3c04711abeb66c9af684149" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643a0fc32e094e1eb52717bd61a2fd85"
I20260923 23:14:25.715987 27036 raft_consensus.cc:3037] T 1064065a85664dc3b7b5c0adada3015f P 643a0fc32e094e1eb52717bd61a2fd85 [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:14:25.715968 26687 leader_election.cc:302] T 1064065a85664dc3b7b5c0adada3015f P b92ee785a3c04711abeb66c9af684149 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08c54376cc9d473596a9b8c014946779, b92ee785a3c04711abeb66c9af684149; no voters:
I20260923 23:14:25.716068 27110 raft_consensus.cc:2781] T 1064065a85664dc3b7b5c0adada3015f P b92ee785a3c04711abeb66c9af684149 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 23:14:25.716238 27110 raft_consensus.cc:686] T 1064065a85664dc3b7b5c0adada3015f P b92ee785a3c04711abeb66c9af684149 [term 1 LEADER]: Becoming Leader. State: Replica: b92ee785a3c04711abeb66c9af684149, State: Running, Role: LEADER
I20260923 23:14:25.716306 27110 consensus_queue.cc:237] T 1064065a85664dc3b7b5c0adada3015f P b92ee785a3c04711abeb66c9af684149 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b92ee785a3c04711abeb66c9af684149" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 36649 } } peers { permanent_uuid: "08c54376cc9d473596a9b8c014946779" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 44665 } } peers { permanent_uuid: "643a0fc32e094e1eb52717bd61a2fd85" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39787 } }
I20260923 23:14:25.716766 27036 raft_consensus.cc:2454] T 1064065a85664dc3b7b5c0adada3015f P 643a0fc32e094e1eb52717bd61a2fd85 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b92ee785a3c04711abeb66c9af684149 in term 1.
I20260923 23:14:25.717022 26599 catalog_manager.cc:5495] T 1064065a85664dc3b7b5c0adada3015f P b92ee785a3c04711abeb66c9af684149 reported cstate change: term changed from 0 to 1, leader changed from <none> to b92ee785a3c04711abeb66c9af684149 (127.24.227.193). New cstate: current_term: 1 leader_uuid: "b92ee785a3c04711abeb66c9af684149" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b92ee785a3c04711abeb66c9af684149" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 36649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "08c54376cc9d473596a9b8c014946779" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 44665 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "643a0fc32e094e1eb52717bd61a2fd85" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39787 } health_report { overall_health: UNKNOWN } } }
Sep 23 23:14:25 dist-test-slave-754n krb5kdc[26554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205264, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.254@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
I20260923 23:14:25.760496 26649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:25.752542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52932 (local address 127.24.227.254:41471)
0923 23:14:25.752622 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:25.752624 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:25.752733 (+ 109us) server_negotiation.cc:407] Connection header received
0923 23:14:25.752811 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:25.752812 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:25.752827 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:25.752885 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:25.753308 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:25.754007 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:25.755165 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:25.755433 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:25.755493 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:25.757853 (+ 2360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:25.757855 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:25.757855 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:25.757861 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:25.759229 (+ 1368us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:25.759628 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:25.759629 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:25.759630 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:25.759679 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:25.759861 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:25.759862 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:25.759862 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:25.760000 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:25.760052 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:25.760238 (+ 186us) server_negotiation.cc:299] Negotiation successful
0923 23:14:25.760346 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260923 23:14:25.760654 27140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:25.752488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41471 (local address 127.0.0.1:52932)
0923 23:14:25.752656 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:25.752668 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:25.752752 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:25.752889 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:25.752889 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:25.753184 (+ 295us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:14:25.753223 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:25.753226 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:25.754128 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:25.754129 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:25.755063 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:25.755066 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:25.755522 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:25.755523 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:25.755593 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:25.756187 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:14:25.756192 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:14:25.757751 (+ 1559us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:14:25.759305 (+ 1554us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:25.759312 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:25.759314 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:25.759551 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:25.759741 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:25.759742 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:25.759742 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:25.759805 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:25.760059 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:14:25.760065 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:14:25.760140 (+ 75us) client_negotiation.cc:769] Sending connection context
0923 23:14:25.760213 (+ 73us) client_negotiation.cc:240] Negotiation successful
0923 23:14:25.760489 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"mutex_wait_us":156,"thread_start_us":78,"threads_started":1}
I20260923 23:14:25.762717 26599 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.24.227.254@KRBTEST.COM'} at 127.0.0.1:52932:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260923 23:14:25.762951 26599 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 23:14:25.764297 26876 tablet_service.cc:1467] Processing CreateTablet for tablet c6140379fa6e417bb707c46c96573c51 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f8570f5d86964ba7acace76f0efdb892]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260923 23:14:25.764302 26736 tablet_service.cc:1467] Processing CreateTablet for tablet c6140379fa6e417bb707c46c96573c51 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f8570f5d86964ba7acace76f0efdb892]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260923 23:14:25.764348 27016 tablet_service.cc:1467] Processing CreateTablet for tablet c6140379fa6e417bb707c46c96573c51 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f8570f5d86964ba7acace76f0efdb892]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260923 23:14:25.764444 26876 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6140379fa6e417bb707c46c96573c51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:14:25.764446 26736 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6140379fa6e417bb707c46c96573c51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:14:25.764500 27016 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6140379fa6e417bb707c46c96573c51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:14:25.765636 27103 tablet_bootstrap.cc:492] T c6140379fa6e417bb707c46c96573c51 P 643a0fc32e094e1eb52717bd61a2fd85: Bootstrap starting.
I20260923 23:14:25.765849 27102 tablet_bootstrap.cc:492] T c6140379fa6e417bb707c46c96573c51 P 08c54376cc9d473596a9b8c014946779: Bootstrap starting.
I20260923 23:14:25.766047 27103 tablet_bootstrap.cc:654] T c6140379fa6e417bb707c46c96573c51 P 643a0fc32e094e1eb52717bd61a2fd85: Neither blocks nor log segments found. Creating new log.
I20260923 23:14:25.766088 27102 tablet_bootstrap.cc:654] T c6140379fa6e417bb707c46c96573c51 P 08c54376cc9d473596a9b8c014946779: Neither blocks nor log segments found. Creating new log.
I20260923 23:14:25.766641 27102 tablet_bootstrap.cc:492] T c6140379fa6e417bb707c46c96573c51 P 08c54376cc9d473596a9b8c014946779: No bootstrap required, opened a new log
I20260923 23:14:25.766674 27102 ts_tablet_manager.cc:1397] T c6140379fa6e417bb707c46c96573c51 P 08c54376cc9d473596a9b8c014946779: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260923 23:14:25.766745 27102 raft_consensus.cc:348] T c6140379fa6e417bb707c46c96573c51 P 08c54376cc9d473596a9b8c014946779 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b92ee785a3c04711abeb66c9af684149" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 36649 } } peers { permanent_uuid: "08c54376cc9d473596a9b8c014946779" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 44665 } } peers { permanent_uuid: "643a0fc32e094e1eb52717bd61a2fd85" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39787 } }
I20260923 23:14:25.766801 27102 raft_consensus.cc:374] T c6140379fa6e417bb707c46c96573c51 P 08c54376cc9d473596a9b8c014946779 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:14:25.766814 27102 raft_consensus.cc:729] T c6140379fa6e417bb707c46c96573c51 P 08c54376cc9d473596a9b8c014946779 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08c54376cc9d473596a9b8c014946779, State: Initialized, Role: FOLLOWER
I20260923 23:14:25.766876 27102 consensus_queue.cc:260] T c6140379fa6e417bb707c46c96573c51 P 08c54376cc9d473596a9b8c014946779 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b92ee785a3c04711abeb66c9af684149" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 36649 } } peers { permanent_uuid: "08c54376cc9d473596a9b8c014946779" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 44665 } } peers { permanent_uuid: "643a0fc32e094e1eb52717bd61a2fd85" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39787 } }
I20260923 23:14:25.766979 27102 ts_tablet_manager.cc:1428] T c6140379fa6e417bb707c46c96573c51 P 08c54376cc9d473596a9b8c014946779: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:25.766996 27122 tablet_replica.cc:439] T c6140379fa6e417bb707c46c96573c51 P 08c54376cc9d473596a9b8c014946779: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b92ee785a3c04711abeb66c9af684149" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 36649 } } peers { permanent_uuid: "08c54376cc9d473596a9b8c014946779" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 44665 } } peers { permanent_uuid: "643a0fc32e094e1eb52717bd61a2fd85" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39787 } } }
I20260923 23:14:25.767062 27122 tablet_replica.cc:442] T c6140379fa6e417bb707c46c96573c51 P 08c54376cc9d473596a9b8c014946779: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 23:14:25.767539 27103 tablet_bootstrap.cc:492] T c6140379fa6e417bb707c46c96573c51 P 643a0fc32e094e1eb52717bd61a2fd85: No bootstrap required, opened a new log
I20260923 23:14:25.767590 27103 ts_tablet_manager.cc:1397] T c6140379fa6e417bb707c46c96573c51 P 643a0fc32e094e1eb52717bd61a2fd85: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 23:14:25.767659 27105 tablet_bootstrap.cc:492] T c6140379fa6e417bb707c46c96573c51 P b92ee785a3c04711abeb66c9af684149: Bootstrap starting.
I20260923 23:14:25.767784 27103 raft_consensus.cc:348] T c6140379fa6e417bb707c46c96573c51 P 643a0fc32e094e1eb52717bd61a2fd85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b92ee785a3c04711abeb66c9af684149" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 36649 } } peers { permanent_uuid: "08c54376cc9d473596a9b8c014946779" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 44665 } } peers { permanent_uuid: "643a0fc32e094e1eb52717bd61a2fd85" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39787 } }
I20260923 23:14:25.767848 27103 raft_consensus.cc:374] T c6140379fa6e417bb707c46c96573c51 P 643a0fc32e094e1eb52717bd61a2fd85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:14:25.767922 27103 raft_consensus.cc:729] T c6140379fa6e417bb707c46c96573c51 P 643a0fc32e094e1eb52717bd61a2fd85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 643a0fc32e094e1eb52717bd61a2fd85, State: Initialized, Role: FOLLOWER
I20260923 23:14:25.767970 27103 consensus_queue.cc:260] T c6140379fa6e417bb707c46c96573c51 P 643a0fc32e094e1eb52717bd61a2fd85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b92ee785a3c04711abeb66c9af684149" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 36649 } } peers { permanent_uuid: "08c54376cc9d473596a9b8c014946779" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 44665 } } peers { permanent_uuid: "643a0fc32e094e1eb52717bd61a2fd85" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39787 } }
I20260923 23:14:25.768054 27103 ts_tablet_manager.cc:1428] T c6140379fa6e417bb707c46c96573c51 P 643a0fc32e094e1eb52717bd61a2fd85: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260923 23:14:25.768088 27105 tablet_bootstrap.cc:654] T c6140379fa6e417bb707c46c96573c51 P b92ee785a3c04711abeb66c9af684149: Neither blocks nor log segments found. Creating new log.
I20260923 23:14:25.768118 27109 tablet_replica.cc:439] T c6140379fa6e417bb707c46c96573c51 P 643a0fc32e094e1eb52717bd61a2fd85: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b92ee785a3c04711abeb66c9af684149" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 36649 } } peers { permanent_uuid: "08c54376cc9d473596a9b8c014946779" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 44665 } } peers { permanent_uuid: "643a0fc32e094e1eb52717bd61a2fd85" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39787 } } }
I20260923 23:14:25.768236 27109 tablet_replica.cc:442] T c6140379fa6e417bb707c46c96573c51 P 643a0fc32e094e1eb52717bd61a2fd85: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 23:14:25.768783 27105 tablet_bootstrap.cc:492] T c6140379fa6e417bb707c46c96573c51 P b92ee785a3c04711abeb66c9af684149: No bootstrap required, opened a new log
I20260923 23:14:25.768837 27105 ts_tablet_manager.cc:1397] T c6140379fa6e417bb707c46c96573c51 P b92ee785a3c04711abeb66c9af684149: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260923 23:14:25.768977 27105 raft_consensus.cc:348] T c6140379fa6e417bb707c46c96573c51 P b92ee785a3c04711abeb66c9af684149 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b92ee785a3c04711abeb66c9af684149" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 36649 } } peers { permanent_uuid: "08c54376cc9d473596a9b8c014946779" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 44665 } } peers { permanent_uuid: "643a0fc32e094e1eb52717bd61a2fd85" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39787 } }
I20260923 23:14:25.769057 27105 raft_consensus.cc:374] T c6140379fa6e417bb707c46c96573c51 P b92ee785a3c04711abeb66c9af684149 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:14:25.769085 27105 raft_consensus.cc:729] T c6140379fa6e417bb707c46c96573c51 P b92ee785a3c04711abeb66c9af684149 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b92ee785a3c04711abeb66c9af684149, State: Initialized, Role: FOLLOWER
I20260923 23:14:25.769134 27105 consensus_queue.cc:260] T c6140379fa6e417bb707c46c96573c51 P b92ee785a3c04711abeb66c9af684149 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b92ee785a3c04711abeb66c9af684149" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 36649 } } peers { permanent_uuid: "08c54376cc9d473596a9b8c014946779" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 44665 } } peers { permanent_uuid: "643a0fc32e094e1eb52717bd61a2fd85" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39787 } }
I20260923 23:14:25.769222 27105 ts_tablet_manager.cc:1428] T c6140379fa6e417bb707c46c96573c51 P b92ee785a3c04711abeb66c9af684149: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260923 23:14:25.769279 27110 tablet_replica.cc:439] T c6140379fa6e417bb707c46c96573c51 P b92ee785a3c04711abeb66c9af684149: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b92ee785a3c04711abeb66c9af684149" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 36649 } } peers { permanent_uuid: "08c54376cc9d473596a9b8c014946779" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 44665 } } peers { permanent_uuid: "643a0fc32e094e1eb52717bd61a2fd85" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39787 } } }
I20260923 23:14:25.769361 27110 tablet_replica.cc:442] T c6140379fa6e417bb707c46c96573c51 P b92ee785a3c04711abeb66c9af684149: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 23:14:25.771237 27109 raft_consensus.cc:482] T c6140379fa6e417bb707c46c96573c51 P 643a0fc32e094e1eb52717bd61a2fd85 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 23:14:25.771297 27109 raft_consensus.cc:504] T c6140379fa6e417bb707c46c96573c51 P 643a0fc32e094e1eb52717bd61a2fd85 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b92ee785a3c04711abeb66c9af684149" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 36649 } } peers { permanent_uuid: "08c54376cc9d473596a9b8c014946779" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 44665 } } peers { permanent_uuid: "643a0fc32e094e1eb52717bd61a2fd85" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39787 } }
I20260923 23:14:25.771522 27109 leader_election.cc:288] T c6140379fa6e417bb707c46c96573c51 P 643a0fc32e094e1eb52717bd61a2fd85 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b92ee785a3c04711abeb66c9af684149 (127.24.227.193:36649), 08c54376cc9d473596a9b8c014946779 (127.24.227.194:44665)
I20260923 23:14:25.775070 27100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:25.771788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.195:42621 (local address 127.24.227.193:36649)
0923 23:14:25.771825 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:25.771827 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:25.771835 (+ 8us) server_negotiation.cc:407] Connection header received
0923 23:14:25.771946 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:25.771948 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:25.771959 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:25.772013 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:14:25.772243 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:25.772681 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:25.774178 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:25.774723 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:25.774795 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:25.774802 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:25.774983 (+ 181us) server_negotiation.cc:299] Negotiation successful
0923 23:14:25.775015 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260923 23:14:25.775070 27099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:25.771698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.195:45239 (local address 127.24.227.194:44665)
0923 23:14:25.771758 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:25.771760 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:25.771866 (+ 106us) server_negotiation.cc:407] Connection header received
0923 23:14:25.771999 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:25.772000 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:25.772009 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:25.772064 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:14:25.772194 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:25.772670 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:25.774178 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:25.774714 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:25.774795 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:25.774802 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:25.774982 (+ 180us) server_negotiation.cc:299] Negotiation successful
0923 23:14:25.775015 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260923 23:14:25.775151 27144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:25.771659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.194:44665 (local address 127.24.227.195:45239)
0923 23:14:25.771794 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:25.771802 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:25.771911 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:25.772062 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:25.772063 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:25.772064 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:14:25.772119 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:25.772121 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:25.772777 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:25.772778 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:25.774091 (+ 1313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:25.774093 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:25.774830 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:25.774830 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:25.774905 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:25.774909 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 23:14:25.774966 (+ 57us) client_negotiation.cc:240] Negotiation successful
0923 23:14:25.775020 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"mutex_wait_us":37,"thread_start_us":67,"threads_started":1}
I20260923 23:14:25.775192 27145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:25.771642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.193:36649 (local address 127.24.227.195:42621)
0923 23:14:25.771777 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:25.771788 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:25.771841 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:25.772132 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:25.772133 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:25.772135 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:14:25.772181 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:25.772182 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:25.772756 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:25.772757 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:25.774091 (+ 1334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:25.774093 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:25.774827 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:25.774829 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:25.774905 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:25.774911 (+ 6us) client_negotiation.cc:769] Sending connection context
0923 23:14:25.774966 (+ 55us) client_negotiation.cc:240] Negotiation successful
0923 23:14:25.775020 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"mutex_wait_us":99,"thread_start_us":53,"threads_started":1}
I20260923 23:14:25.775326 26896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6140379fa6e417bb707c46c96573c51" candidate_uuid: "643a0fc32e094e1eb52717bd61a2fd85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08c54376cc9d473596a9b8c014946779" is_pre_election: true
I20260923 23:14:25.775441 26896 raft_consensus.cc:2454] T c6140379fa6e417bb707c46c96573c51 P 08c54376cc9d473596a9b8c014946779 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 643a0fc32e094e1eb52717bd61a2fd85 in term 0.
I20260923 23:14:25.775440 26756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6140379fa6e417bb707c46c96573c51" candidate_uuid: "643a0fc32e094e1eb52717bd61a2fd85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b92ee785a3c04711abeb66c9af684149" is_pre_election: true
I20260923 23:14:25.775528 26756 raft_consensus.cc:2454] T c6140379fa6e417bb707c46c96573c51 P b92ee785a3c04711abeb66c9af684149 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 643a0fc32e094e1eb52717bd61a2fd85 in term 0.
I20260923 23:14:25.775663 26967 leader_election.cc:302] T c6140379fa6e417bb707c46c96573c51 P 643a0fc32e094e1eb52717bd61a2fd85 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08c54376cc9d473596a9b8c014946779, 643a0fc32e094e1eb52717bd61a2fd85; no voters:
I20260923 23:14:25.775842 27109 raft_consensus.cc:2781] T c6140379fa6e417bb707c46c96573c51 P 643a0fc32e094e1eb52717bd61a2fd85 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 23:14:25.775928 27109 raft_consensus.cc:482] T c6140379fa6e417bb707c46c96573c51 P 643a0fc32e094e1eb52717bd61a2fd85 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 23:14:25.775961 27109 raft_consensus.cc:3037] T c6140379fa6e417bb707c46c96573c51 P 643a0fc32e094e1eb52717bd61a2fd85 [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:14:25.776635 27109 raft_consensus.cc:504] T c6140379fa6e417bb707c46c96573c51 P 643a0fc32e094e1eb52717bd61a2fd85 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b92ee785a3c04711abeb66c9af684149" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 36649 } } peers { permanent_uuid: "08c54376cc9d473596a9b8c014946779" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 44665 } } peers { permanent_uuid: "643a0fc32e094e1eb52717bd61a2fd85" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39787 } }
I20260923 23:14:25.776767 27109 leader_election.cc:288] T c6140379fa6e417bb707c46c96573c51 P 643a0fc32e094e1eb52717bd61a2fd85 [CANDIDATE]: Term 1 election: Requested vote from peers b92ee785a3c04711abeb66c9af684149 (127.24.227.193:36649), 08c54376cc9d473596a9b8c014946779 (127.24.227.194:44665)
I20260923 23:14:25.776911 26756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6140379fa6e417bb707c46c96573c51" candidate_uuid: "643a0fc32e094e1eb52717bd61a2fd85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b92ee785a3c04711abeb66c9af684149"
I20260923 23:14:25.776923 26896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6140379fa6e417bb707c46c96573c51" candidate_uuid: "643a0fc32e094e1eb52717bd61a2fd85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08c54376cc9d473596a9b8c014946779"
I20260923 23:14:25.776973 26756 raft_consensus.cc:3037] T c6140379fa6e417bb707c46c96573c51 P b92ee785a3c04711abeb66c9af684149 [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:14:25.776973 26896 raft_consensus.cc:3037] T c6140379fa6e417bb707c46c96573c51 P 08c54376cc9d473596a9b8c014946779 [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:14:25.777556 26756 raft_consensus.cc:2454] T c6140379fa6e417bb707c46c96573c51 P b92ee785a3c04711abeb66c9af684149 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 643a0fc32e094e1eb52717bd61a2fd85 in term 1.
I20260923 23:14:25.777555 26896 raft_consensus.cc:2454] T c6140379fa6e417bb707c46c96573c51 P 08c54376cc9d473596a9b8c014946779 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 643a0fc32e094e1eb52717bd61a2fd85 in term 1.
I20260923 23:14:25.777714 26967 leader_election.cc:302] T c6140379fa6e417bb707c46c96573c51 P 643a0fc32e094e1eb52717bd61a2fd85 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08c54376cc9d473596a9b8c014946779, 643a0fc32e094e1eb52717bd61a2fd85; no voters:
I20260923 23:14:25.777801 27109 raft_consensus.cc:2781] T c6140379fa6e417bb707c46c96573c51 P 643a0fc32e094e1eb52717bd61a2fd85 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 23:14:25.777901 27109 raft_consensus.cc:686] T c6140379fa6e417bb707c46c96573c51 P 643a0fc32e094e1eb52717bd61a2fd85 [term 1 LEADER]: Becoming Leader. State: Replica: 643a0fc32e094e1eb52717bd61a2fd85, State: Running, Role: LEADER
I20260923 23:14:25.777961 27109 consensus_queue.cc:237] T c6140379fa6e417bb707c46c96573c51 P 643a0fc32e094e1eb52717bd61a2fd85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b92ee785a3c04711abeb66c9af684149" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 36649 } } peers { permanent_uuid: "08c54376cc9d473596a9b8c014946779" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 44665 } } peers { permanent_uuid: "643a0fc32e094e1eb52717bd61a2fd85" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39787 } }
I20260923 23:14:25.778280 27146 tablet_replica.cc:439] T c6140379fa6e417bb707c46c96573c51 P 643a0fc32e094e1eb52717bd61a2fd85: TxnStatusTablet state changed. Reason: New leader 643a0fc32e094e1eb52717bd61a2fd85. Latest consensus state: current_term: 1 leader_uuid: "643a0fc32e094e1eb52717bd61a2fd85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b92ee785a3c04711abeb66c9af684149" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 36649 } } peers { permanent_uuid: "08c54376cc9d473596a9b8c014946779" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 44665 } } peers { permanent_uuid: "643a0fc32e094e1eb52717bd61a2fd85" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39787 } } }
I20260923 23:14:25.778371 27146 tablet_replica.cc:442] T c6140379fa6e417bb707c46c96573c51 P 643a0fc32e094e1eb52717bd61a2fd85: This TxnStatusTablet replica's current role is: LEADER
I20260923 23:14:25.778528 27148 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260923 23:14:25.778533 26599 catalog_manager.cc:5495] T c6140379fa6e417bb707c46c96573c51 P 643a0fc32e094e1eb52717bd61a2fd85 reported cstate change: term changed from 0 to 1, leader changed from <none> to 643a0fc32e094e1eb52717bd61a2fd85 (127.24.227.195). New cstate: current_term: 1 leader_uuid: "643a0fc32e094e1eb52717bd61a2fd85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b92ee785a3c04711abeb66c9af684149" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 36649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "08c54376cc9d473596a9b8c014946779" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 44665 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "643a0fc32e094e1eb52717bd61a2fd85" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39787 } health_report { overall_health: HEALTHY } } }
I20260923 23:14:25.778668 27148 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260923 23:14:25.783699 27101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:25.781320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33464 (local address 127.24.227.195:39787)
0923 23:14:25.781359 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:25.781361 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:25.781367 (+ 6us) server_negotiation.cc:407] Connection header received
0923 23:14:25.781389 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:25.781389 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:25.781401 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:25.781470 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 23:14:25.781616 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:25.782032 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:25.783028 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:25.783196 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:25.783243 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:25.783439 (+ 196us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 23:14:25.783513 (+ 74us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 23:14:25.783563 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:25.783645 (+ 82us) server_negotiation.cc:299] Negotiation successful
0923 23:14:25.783658 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 23:14:25.783756 27140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:25.781121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.195:39787 (local address 127.0.0.1:33464)
0923 23:14:25.781183 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:25.781194 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:25.781239 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:25.781504 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:25.781505 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:25.781507 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0923 23:14:25.781547 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:25.781550 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:25.782113 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:25.782114 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:25.782948 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:25.782950 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:25.783271 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:25.783273 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:25.783337 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:25.783353 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0923 23:14:25.783576 (+ 223us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0923 23:14:25.783584 (+ 8us) client_negotiation.cc:769] Sending connection context
0923 23:14:25.783642 (+ 58us) client_negotiation.cc:240] Negotiation successful
0923 23:14:25.783688 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260923 23:14:25.784628 26756 raft_consensus.cc:1261] T c6140379fa6e417bb707c46c96573c51 P b92ee785a3c04711abeb66c9af684149 [term 1 FOLLOWER]: Refusing update from remote peer 643a0fc32e094e1eb52717bd61a2fd85: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 23:14:25.784704 26896 raft_consensus.cc:1261] T c6140379fa6e417bb707c46c96573c51 P 08c54376cc9d473596a9b8c014946779 [term 1 FOLLOWER]: Refusing update from remote peer 643a0fc32e094e1eb52717bd61a2fd85: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 23:14:25.784845 27146 consensus_queue.cc:1035] T c6140379fa6e417bb707c46c96573c51 P 643a0fc32e094e1eb52717bd61a2fd85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b92ee785a3c04711abeb66c9af684149" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 36649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 23:14:25.784960 27109 consensus_queue.cc:1035] T c6140379fa6e417bb707c46c96573c51 P 643a0fc32e094e1eb52717bd61a2fd85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "08c54376cc9d473596a9b8c014946779" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 44665 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 23:14:25.785495 27122 tablet_replica.cc:439] T c6140379fa6e417bb707c46c96573c51 P 08c54376cc9d473596a9b8c014946779: TxnStatusTablet state changed. Reason: New leader 643a0fc32e094e1eb52717bd61a2fd85. Latest consensus state: current_term: 1 leader_uuid: "643a0fc32e094e1eb52717bd61a2fd85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b92ee785a3c04711abeb66c9af684149" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 36649 } } peers { permanent_uuid: "08c54376cc9d473596a9b8c014946779" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 44665 } } peers { permanent_uuid: "643a0fc32e094e1eb52717bd61a2fd85" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39787 } } }
I20260923 23:14:25.785523 27110 tablet_replica.cc:439] T c6140379fa6e417bb707c46c96573c51 P b92ee785a3c04711abeb66c9af684149: TxnStatusTablet state changed. Reason: New leader 643a0fc32e094e1eb52717bd61a2fd85. Latest consensus state: current_term: 1 leader_uuid: "643a0fc32e094e1eb52717bd61a2fd85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b92ee785a3c04711abeb66c9af684149" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 36649 } } peers { permanent_uuid: "08c54376cc9d473596a9b8c014946779" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 44665 } } peers { permanent_uuid: "643a0fc32e094e1eb52717bd61a2fd85" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39787 } } }
I20260923 23:14:25.785570 27122 tablet_replica.cc:442] T c6140379fa6e417bb707c46c96573c51 P 08c54376cc9d473596a9b8c014946779: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 23:14:25.785594 27110 tablet_replica.cc:442] T c6140379fa6e417bb707c46c96573c51 P b92ee785a3c04711abeb66c9af684149: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 23:14:25.786329 27109 tablet_replica.cc:439] T c6140379fa6e417bb707c46c96573c51 P 643a0fc32e094e1eb52717bd61a2fd85: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "643a0fc32e094e1eb52717bd61a2fd85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b92ee785a3c04711abeb66c9af684149" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 36649 } } peers { permanent_uuid: "08c54376cc9d473596a9b8c014946779" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 44665 } } peers { permanent_uuid: "643a0fc32e094e1eb52717bd61a2fd85" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39787 } } }
I20260923 23:14:25.786414 27109 tablet_replica.cc:442] T c6140379fa6e417bb707c46c96573c51 P 643a0fc32e094e1eb52717bd61a2fd85: This TxnStatusTablet replica's current role is: LEADER
I20260923 23:14:25.787035 27122 tablet_replica.cc:439] T c6140379fa6e417bb707c46c96573c51 P 08c54376cc9d473596a9b8c014946779: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "643a0fc32e094e1eb52717bd61a2fd85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b92ee785a3c04711abeb66c9af684149" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 36649 } } peers { permanent_uuid: "08c54376cc9d473596a9b8c014946779" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 44665 } } peers { permanent_uuid: "643a0fc32e094e1eb52717bd61a2fd85" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39787 } } }
I20260923 23:14:25.787118 27122 tablet_replica.cc:442] T c6140379fa6e417bb707c46c96573c51 P 08c54376cc9d473596a9b8c014946779: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 23:14:25.788434 27146 tablet_replica.cc:439] T c6140379fa6e417bb707c46c96573c51 P 643a0fc32e094e1eb52717bd61a2fd85: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "643a0fc32e094e1eb52717bd61a2fd85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b92ee785a3c04711abeb66c9af684149" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 36649 } } peers { permanent_uuid: "08c54376cc9d473596a9b8c014946779" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 44665 } } peers { permanent_uuid: "643a0fc32e094e1eb52717bd61a2fd85" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39787 } } }
I20260923 23:14:25.788508 27146 tablet_replica.cc:442] T c6140379fa6e417bb707c46c96573c51 P 643a0fc32e094e1eb52717bd61a2fd85: This TxnStatusTablet replica's current role is: LEADER
I20260923 23:14:25.788957 27117 mvcc.cc:205] Tried to move back new op lower bound from 7332680768652206080 to 7332680768626929664. Current Snapshot: MvccSnapshot[applied={T|T < 7332680768652206080}]
I20260923 23:14:25.789049 27110 tablet_replica.cc:439] T c6140379fa6e417bb707c46c96573c51 P b92ee785a3c04711abeb66c9af684149: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "643a0fc32e094e1eb52717bd61a2fd85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b92ee785a3c04711abeb66c9af684149" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 36649 } } peers { permanent_uuid: "08c54376cc9d473596a9b8c014946779" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 44665 } } peers { permanent_uuid: "643a0fc32e094e1eb52717bd61a2fd85" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39787 } } }
I20260923 23:14:25.789124 27110 tablet_replica.cc:442] T c6140379fa6e417bb707c46c96573c51 P b92ee785a3c04711abeb66c9af684149: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 23:14:25.790546 27100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:25.787772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60910 (local address 127.24.227.193:36649)
0923 23:14:25.787805 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:25.787807 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:25.787846 (+ 39us) server_negotiation.cc:407] Connection header received
0923 23:14:25.787966 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:25.787967 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:25.787988 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:25.788051 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 23:14:25.788177 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:25.788878 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:25.789796 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:25.790097 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:25.790175 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:25.790271 (+ 96us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 23:14:25.790379 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 23:14:25.790446 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:25.790468 (+ 22us) server_negotiation.cc:299] Negotiation successful
0923 23:14:25.790486 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260923 23:14:25.795118 27101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:25.792765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.193:57823 (local address 127.24.227.195:39787)
0923 23:14:25.792799 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:25.792801 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:25.792904 (+ 103us) server_negotiation.cc:407] Connection header received
0923 23:14:25.792926 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:25.792927 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:25.792935 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:25.792999 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 23:14:25.793090 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:25.793758 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:25.794432 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:25.794717 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:25.794785 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:25.794927 (+ 142us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 23:14:25.794954 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 23:14:25.795001 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:25.795040 (+ 39us) server_negotiation.cc:299] Negotiation successful
0923 23:14:25.795054 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260923 23:14:25.795123 27156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:25.792682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.195:39787 (local address 127.24.227.193:57823)
0923 23:14:25.792824 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:25.792834 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:25.792895 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:25.793001 (+ 106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:25.793001 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:25.793002 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0923 23:14:25.793053 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:25.793054 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:25.793831 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:25.793831 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:25.794360 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:25.794361 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:25.794790 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:25.794791 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:25.794836 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:25.794846 (+ 10us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0923 23:14:25.795006 (+ 160us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0923 23:14:25.795007 (+ 1us) client_negotiation.cc:769] Sending connection context
0923 23:14:25.795037 (+ 30us) client_negotiation.cc:240] Negotiation successful
0923 23:14:25.795064 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":88,"threads_started":1}
I20260923 23:14:25.797062 26896 raft_consensus.cc:1261] T 1064065a85664dc3b7b5c0adada3015f P 08c54376cc9d473596a9b8c014946779 [term 1 FOLLOWER]: Refusing update from remote peer b92ee785a3c04711abeb66c9af684149: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 23:14:25.797071 27036 raft_consensus.cc:1261] T 1064065a85664dc3b7b5c0adada3015f P 643a0fc32e094e1eb52717bd61a2fd85 [term 1 FOLLOWER]: Refusing update from remote peer b92ee785a3c04711abeb66c9af684149: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 23:14:25.797305 27110 consensus_queue.cc:1035] T 1064065a85664dc3b7b5c0adada3015f P b92ee785a3c04711abeb66c9af684149 [LEADER]: Connected to new peer: Peer: permanent_uuid: "643a0fc32e094e1eb52717bd61a2fd85" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 23:14:25.797451 27133 consensus_queue.cc:1035] T 1064065a85664dc3b7b5c0adada3015f P b92ee785a3c04711abeb66c9af684149 [LEADER]: Connected to new peer: Peer: permanent_uuid: "08c54376cc9d473596a9b8c014946779" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 44665 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 23:14:25.805189 27162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:25.802837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.193:36649 (local address 127.24.227.195:42375)
0923 23:14:25.802969 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:25.802977 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:25.803054 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:25.803210 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:25.803210 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:25.803212 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0923 23:14:25.803267 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:25.803268 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:25.803826 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:25.803827 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:25.804432 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:25.804434 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:25.804849 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:25.804850 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:25.804911 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:25.804924 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0923 23:14:25.805069 (+ 145us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0923 23:14:25.805070 (+ 1us) client_negotiation.cc:769] Sending connection context
0923 23:14:25.805095 (+ 25us) client_negotiation.cc:240] Negotiation successful
0923 23:14:25.805128 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":69,"threads_started":1}
I20260923 23:14:25.805187 27100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:25.802875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.195:42375 (local address 127.24.227.193:36649)
0923 23:14:25.802913 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:25.802915 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:25.803044 (+ 129us) server_negotiation.cc:407] Connection header received
0923 23:14:25.803148 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:25.803149 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:25.803157 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:25.803206 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 23:14:25.803335 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:25.803749 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:25.804549 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:25.804763 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:25.804846 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:25.804987 (+ 141us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 23:14:25.805019 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 23:14:25.805066 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:25.805112 (+ 46us) server_negotiation.cc:299] Negotiation successful
0923 23:14:25.805129 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260923 23:14:25.815850 26599 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52924:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 23:14:25.815958 26599 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52924:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 23:14:25.816524 26599 catalog_manager.cc:5782] T 00000000000000000000000000000000 P edb11fcb0fbd4223a79b05d5bfd4e8da: Sending DeleteTablet for 3 replicas of tablet 1064065a85664dc3b7b5c0adada3015f
I20260923 23:14:25.816782 26736 tablet_service.cc:1514] Processing DeleteTablet for tablet 1064065a85664dc3b7b5c0adada3015f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 23:14:25 UTC) from {username='kudu', principal='kudu/127.24.227.254@KRBTEST.COM'} at 127.0.0.1:60900
I20260923 23:14:25.816812 27016 tablet_service.cc:1514] Processing DeleteTablet for tablet 1064065a85664dc3b7b5c0adada3015f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 23:14:25 UTC) from {username='kudu', principal='kudu/127.24.227.254@KRBTEST.COM'} at 127.0.0.1:33452
I20260923 23:14:25.816869 27125 tablet_replica.cc:330] T 1064065a85664dc3b7b5c0adada3015f P b92ee785a3c04711abeb66c9af684149: stopping tablet replica
I20260923 23:14:25.816890 27127 tablet_replica.cc:330] T 1064065a85664dc3b7b5c0adada3015f P 643a0fc32e094e1eb52717bd61a2fd85: stopping tablet replica
I20260923 23:14:25.816916 27125 raft_consensus.cc:2229] T 1064065a85664dc3b7b5c0adada3015f P b92ee785a3c04711abeb66c9af684149 [term 1 LEADER]: Raft consensus shutting down.
I20260923 23:14:25.816941 27127 raft_consensus.cc:2229] T 1064065a85664dc3b7b5c0adada3015f P 643a0fc32e094e1eb52717bd61a2fd85 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 23:14:25.816984 27127 raft_consensus.cc:2258] T 1064065a85664dc3b7b5c0adada3015f P 643a0fc32e094e1eb52717bd61a2fd85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 23:14:25.816991 27125 raft_consensus.cc:2258] T 1064065a85664dc3b7b5c0adada3015f P b92ee785a3c04711abeb66c9af684149 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 23:14:25.817061 26876 tablet_service.cc:1514] Processing DeleteTablet for tablet 1064065a85664dc3b7b5c0adada3015f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 23:14:25 UTC) from {username='kudu', principal='kudu/127.24.227.254@KRBTEST.COM'} at 127.0.0.1:35486
I20260923 23:14:25.817203 27126 tablet_replica.cc:330] T 1064065a85664dc3b7b5c0adada3015f P 08c54376cc9d473596a9b8c014946779: stopping tablet replica
I20260923 23:14:25.817284 27126 raft_consensus.cc:2229] T 1064065a85664dc3b7b5c0adada3015f P 08c54376cc9d473596a9b8c014946779 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 23:14:25.817298 27127 ts_tablet_manager.cc:1905] T 1064065a85664dc3b7b5c0adada3015f P 643a0fc32e094e1eb52717bd61a2fd85: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 23:14:25.817323 27126 raft_consensus.cc:2258] T 1064065a85664dc3b7b5c0adada3015f P 08c54376cc9d473596a9b8c014946779 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 23:14:25.817504 27125 ts_tablet_manager.cc:1905] T 1064065a85664dc3b7b5c0adada3015f P b92ee785a3c04711abeb66c9af684149: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 23:14:25.817664 27126 ts_tablet_manager.cc:1905] T 1064065a85664dc3b7b5c0adada3015f P 08c54376cc9d473596a9b8c014946779: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 23:14:25.819023 27127 ts_tablet_manager.cc:1918] T 1064065a85664dc3b7b5c0adada3015f P 643a0fc32e094e1eb52717bd61a2fd85: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 23:14:25.819094 27127 log.cc:1196] T 1064065a85664dc3b7b5c0adada3015f P 643a0fc32e094e1eb52717bd61a2fd85: Deleting WAL directory at /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/minicluster-data/ts-2/wal/wals/1064065a85664dc3b7b5c0adada3015f
I20260923 23:14:25.819124 27125 ts_tablet_manager.cc:1918] T 1064065a85664dc3b7b5c0adada3015f P b92ee785a3c04711abeb66c9af684149: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 23:14:25.819131 27126 ts_tablet_manager.cc:1918] T 1064065a85664dc3b7b5c0adada3015f P 08c54376cc9d473596a9b8c014946779: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 23:14:25.819178 27126 log.cc:1196] T 1064065a85664dc3b7b5c0adada3015f P 08c54376cc9d473596a9b8c014946779: Deleting WAL directory at /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/minicluster-data/ts-1/wal/wals/1064065a85664dc3b7b5c0adada3015f
I20260923 23:14:25.819178 27125 log.cc:1196] T 1064065a85664dc3b7b5c0adada3015f P b92ee785a3c04711abeb66c9af684149: Deleting WAL directory at /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790205259943518-25487-0/minicluster-data/ts-0/wal/wals/1064065a85664dc3b7b5c0adada3015f
I20260923 23:14:25.819394 27125 ts_tablet_manager.cc:1939] T 1064065a85664dc3b7b5c0adada3015f P b92ee785a3c04711abeb66c9af684149: Deleting consensus metadata
I20260923 23:14:25.819394 27126 ts_tablet_manager.cc:1939] T 1064065a85664dc3b7b5c0adada3015f P 08c54376cc9d473596a9b8c014946779: Deleting consensus metadata
I20260923 23:14:25.819478 27127 ts_tablet_manager.cc:1939] T 1064065a85664dc3b7b5c0adada3015f P 643a0fc32e094e1eb52717bd61a2fd85: Deleting consensus metadata
I20260923 23:14:25.819763 26585 catalog_manager.cc:4841] TS 643a0fc32e094e1eb52717bd61a2fd85 (127.24.227.195:39787): tablet 1064065a85664dc3b7b5c0adada3015f (table test-table [id=a0b08e59c4ab42408c3115f35b5fc5a3]) successfully deleted
I20260923 23:14:25.819893 26583 catalog_manager.cc:4841] TS 08c54376cc9d473596a9b8c014946779 (127.24.227.194:44665): tablet 1064065a85664dc3b7b5c0adada3015f (table test-table [id=a0b08e59c4ab42408c3115f35b5fc5a3]) successfully deleted
I20260923 23:14:25.820209 26586 catalog_manager.cc:4841] TS b92ee785a3c04711abeb66c9af684149 (127.24.227.193:36649): tablet 1064065a85664dc3b7b5c0adada3015f (table test-table [id=a0b08e59c4ab42408c3115f35b5fc5a3]) successfully deleted
Sep 23 23:14:25 dist-test-slave-754n krb5kdc[26554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205265, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.24.227.193@KRBTEST.COM
I20260923 23:14:25.836299 27100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:25.826882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60926 (local address 127.24.227.193:36649)
0923 23:14:25.826929 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:25.826931 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:25.827080 (+ 149us) server_negotiation.cc:407] Connection header received
0923 23:14:25.827108 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:25.827109 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:25.827118 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:25.827186 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:25.827790 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:25.828465 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:25.829512 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:25.829817 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:25.829895 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:25.833076 (+ 3181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:25.833089 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:25.833089 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:25.833096 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:25.834920 (+ 1824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:25.835314 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:25.835316 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:25.835317 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:25.835394 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:25.835601 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:25.835601 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:25.835602 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:25.835749 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:25.835911 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:25.836028 (+ 117us) server_negotiation.cc:299] Negotiation successful
0923 23:14:25.836106 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
W20260923 23:14:25.836884 26796 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60926
I20260923 23:14:25.845018 26649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:25.837771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52942 (local address 127.24.227.254:41471)
0923 23:14:25.837812 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:25.837813 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:25.837994 (+ 181us) server_negotiation.cc:407] Connection header received
0923 23:14:25.838043 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:25.838044 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:25.838056 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:25.838113 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:25.838755 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:25.839512 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:25.840550 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:25.840857 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:25.840927 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:25.842210 (+ 1283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:25.842211 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:25.842212 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:25.842219 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:25.843824 (+ 1605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:25.844220 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:25.844221 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:25.844221 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:25.844288 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:25.844506 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:25.844507 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:25.844507 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:25.844642 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:25.844741 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:25.844847 (+ 106us) server_negotiation.cc:299] Negotiation successful
0923 23:14:25.844919 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
W20260923 23:14:25.845181 26599 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52942
I20260923 23:14:25.845934 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 26671
I20260923 23:14:25.851840 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 26811
I20260923 23:14:25.861122 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 26951
I20260923 23:14:25.865906 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 26567
2026-09-23T23:14:25Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2307 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 23:14:25 dist-test-slave-754n krb5kdc[27171](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 23:14:25 dist-test-slave-754n krb5kdc[27171](info): set up 2 sockets
Sep 23 23:14:25 dist-test-slave-754n krb5kdc[27171](info): commencing operation
krb5kdc: starting...
W20260923 23:14:26.879082 25487 mini_kdc.cc:121] Time spent starting KDC: real 0.993s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/krb5kdc/test-user.keytab.
Sep 23 23:14:26 dist-test-slave-754n krb5kdc[27171](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205266, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T23:14:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T23:14:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.227.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.227.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.227.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab.
Entry for principal kudu/127.24.227.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab.
I20260923 23:14:26.983497 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.254:42395
--webserver_interface=127.24.227.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab
--principal=kudu/127.24.227.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:34703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.227.254:42395
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:14:27.059964 27184 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:14:27.060181 27184 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:14:27.060216 27184 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:14:27.061327 27184 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 23:14:27.061389 27184 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260923 23:14:27.061410 27184 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:14:27.061430 27184 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:14:27.061460 27184 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 23:14:27.061484 27184 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 23:14:27.062860 27184 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:34703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.227.254:42395
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab
--principal=kudu/127.24.227.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.254:42395
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.227.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27184
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:14:27.063112 27184 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:14:27.063325 27184 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:14:27.067649 27189 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:27.067679 27190 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:27.067670 27192 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:14:27.067830 27184 server_base.cc:1034] running on GCE node
I20260923 23:14:27.068079 27184 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:14:27.068346 27184 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:14:27.069500 27184 hybrid_clock.cc:648] HybridClock initialized: now 1790205267069492 us; error 36 us; skew 500 ppm
Sep 23 23:14:27 dist-test-slave-754n krb5kdc[27171](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205267, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:14:27.071700 27184 init.cc:377] Logged in from keytab as kudu/127.24.227.254@KRBTEST.COM (short username kudu)
I20260923 23:14:27.072141 27184 webserver.cc:460] Webserver started at http://127.24.227.254:46703/ using document root <none> and password file <none>
I20260923 23:14:27.072343 27184 fs_manager.cc:359] Metadata directory not provided
I20260923 23:14:27.072404 27184 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:14:27.072553 27184 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:14:27.073477 27184 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/minicluster-data/master-0/data/instance:
uuid: "e6ed2aa900084446a84ca867a45396ae"
format_stamp: "Formatted at 2026-09-23 23:14:27 on dist-test-slave-754n"
server_key: "e1c26487290ff0da73f4e920be982584"
server_key_iv: "6a179595bfd6b5e1a2e17a1ce3814906"
server_key_version: "encryptionkey@0"
I20260923 23:14:27.073812 27184 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/minicluster-data/master-0/wal/instance:
uuid: "e6ed2aa900084446a84ca867a45396ae"
format_stamp: "Formatted at 2026-09-23 23:14:27 on dist-test-slave-754n"
server_key: "e1c26487290ff0da73f4e920be982584"
server_key_iv: "6a179595bfd6b5e1a2e17a1ce3814906"
server_key_version: "encryptionkey@0"
I20260923 23:14:27.075016 27184 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 23:14:27.075678 27199 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:27.075843 27184 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 23:14:27.075959 27184 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/minicluster-data/master-0/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/minicluster-data/master-0/wal
uuid: "e6ed2aa900084446a84ca867a45396ae"
format_stamp: "Formatted at 2026-09-23 23:14:27 on dist-test-slave-754n"
server_key: "e1c26487290ff0da73f4e920be982584"
server_key_iv: "6a179595bfd6b5e1a2e17a1ce3814906"
server_key_version: "encryptionkey@0"
I20260923 23:14:27.076042 27184 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:14:27.094691 27184 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:14:27.096441 27184 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:14:27.096589 27184 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:14:27.100936 27184 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.254:42395
I20260923 23:14:27.100976 27261 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.254:42395 every 8 connection(s)
I20260923 23:14:27.101297 27184 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
I20260923 23:14:27.101867 27262 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:14:27.103906 27262 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6ed2aa900084446a84ca867a45396ae: Bootstrap starting.
I20260923 23:14:27.104389 27262 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e6ed2aa900084446a84ca867a45396ae: Neither blocks nor log segments found. Creating new log.
I20260923 23:14:27.104601 27262 log.cc:824] T 00000000000000000000000000000000 P e6ed2aa900084446a84ca867a45396ae: Log is configured to *not* fsync() on all Append() calls
I20260923 23:14:27.105252 27262 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6ed2aa900084446a84ca867a45396ae: No bootstrap required, opened a new log
I20260923 23:14:27.107185 27262 raft_consensus.cc:348] T 00000000000000000000000000000000 P e6ed2aa900084446a84ca867a45396ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6ed2aa900084446a84ca867a45396ae" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 42395 } }
I20260923 23:14:27.107318 27262 raft_consensus.cc:374] T 00000000000000000000000000000000 P e6ed2aa900084446a84ca867a45396ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:14:27.107306 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 27184
I20260923 23:14:27.107339 27262 raft_consensus.cc:729] T 00000000000000000000000000000000 P e6ed2aa900084446a84ca867a45396ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6ed2aa900084446a84ca867a45396ae, State: Initialized, Role: FOLLOWER
I20260923 23:14:27.107416 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/minicluster-data/master-0/wal/instance
I20260923 23:14:27.107497 27262 consensus_queue.cc:260] T 00000000000000000000000000000000 P e6ed2aa900084446a84ca867a45396ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6ed2aa900084446a84ca867a45396ae" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 42395 } }
I20260923 23:14:27.107599 27262 raft_consensus.cc:388] T 00000000000000000000000000000000 P e6ed2aa900084446a84ca867a45396ae [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 23:14:27.107635 27262 raft_consensus.cc:482] T 00000000000000000000000000000000 P e6ed2aa900084446a84ca867a45396ae [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 23:14:27.107640 25487 external_mini_cluster.cc:1428] Setting key cbe84ead0325daf059dec30a94b20fae
I20260923 23:14:27.107683 27262 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e6ed2aa900084446a84ca867a45396ae [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:14:27.108403 27262 raft_consensus.cc:504] T 00000000000000000000000000000000 P e6ed2aa900084446a84ca867a45396ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6ed2aa900084446a84ca867a45396ae" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 42395 } }
I20260923 23:14:27.108489 27262 leader_election.cc:302] T 00000000000000000000000000000000 P e6ed2aa900084446a84ca867a45396ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e6ed2aa900084446a84ca867a45396ae; no voters:
I20260923 23:14:27.108644 27262 leader_election.cc:288] T 00000000000000000000000000000000 P e6ed2aa900084446a84ca867a45396ae [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 23:14:27.108667 27267 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e6ed2aa900084446a84ca867a45396ae [term 1 FOLLOWER]: Leader election won for term 1
I20260923 23:14:27.108762 27267 raft_consensus.cc:686] T 00000000000000000000000000000000 P e6ed2aa900084446a84ca867a45396ae [term 1 LEADER]: Becoming Leader. State: Replica: e6ed2aa900084446a84ca867a45396ae, State: Running, Role: LEADER
I20260923 23:14:27.108834 27267 consensus_queue.cc:237] T 00000000000000000000000000000000 P e6ed2aa900084446a84ca867a45396ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6ed2aa900084446a84ca867a45396ae" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 42395 } }
I20260923 23:14:27.108870 27262 sys_catalog.cc:564] T 00000000000000000000000000000000 P e6ed2aa900084446a84ca867a45396ae [sys.catalog]: configured and running, proceeding with master startup.
I20260923 23:14:27.109145 27267 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6ed2aa900084446a84ca867a45396ae [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e6ed2aa900084446a84ca867a45396ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6ed2aa900084446a84ca867a45396ae" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 42395 } } }
I20260923 23:14:27.109220 27267 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6ed2aa900084446a84ca867a45396ae [sys.catalog]: This master's current role is: LEADER
I20260923 23:14:27.109346 27267 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6ed2aa900084446a84ca867a45396ae [sys.catalog]: SysCatalogTable state changed. Reason: New leader e6ed2aa900084446a84ca867a45396ae. Latest consensus state: current_term: 1 leader_uuid: "e6ed2aa900084446a84ca867a45396ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6ed2aa900084446a84ca867a45396ae" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 42395 } } }
I20260923 23:14:27.109416 27267 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6ed2aa900084446a84ca867a45396ae [sys.catalog]: This master's current role is: LEADER
I20260923 23:14:27.109719 27272 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 23:14:27.110231 27272 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 23:14:27.111801 27272 catalog_manager.cc:1347] Generated new cluster ID: 9f843153348b404daa56dfd8bd80c4f1
I20260923 23:14:27.111852 27272 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 23:14:27 dist-test-slave-754n krb5kdc[27171](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205266, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
I20260923 23:14:27.118117 27266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:27.108087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50644 (local address 127.24.227.254:42395)
0923 23:14:27.108255 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:27.108262 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:27.108295 (+ 33us) server_negotiation.cc:407] Connection header received
0923 23:14:27.108378 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:27.108383 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:27.108418 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:27.108560 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:27.109469 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:27.110842 (+ 1373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:27.111816 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:27.112303 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:27.112417 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:27.114601 (+ 2184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:27.114605 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:27.114606 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:27.114616 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:27.116678 (+ 2062us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:27.117030 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:27.117032 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:27.117033 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:27.117126 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:27.117325 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:27.117326 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:27.117327 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:27.117540 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:27.117688 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:27.117798 (+ 110us) server_negotiation.cc:299] Negotiation successful
0923 23:14:27.117925 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":98,"threads_started":1}
I20260923 23:14:27.156661 27272 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 23:14:27.157155 27272 catalog_manager.cc:1504] Loading token signing keys...
I20260923 23:14:27.163702 27272 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e6ed2aa900084446a84ca867a45396ae: Generated new TSK 0
I20260923 23:14:27.163889 27272 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.227.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.227.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.227.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.193.keytab.
Entry for principal kudu/127.24.227.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.193.keytab.
I20260923 23:14:27.194346 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.227.193:0
--local_ip_for_outbound_sockets=127.24.227.193
--webserver_interface=127.24.227.193
--webserver_port=0
--tserver_master_addrs=127.24.227.254:42395
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.193.keytab
--principal=kudu/127.24.227.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:34703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:14:27.272385 27288 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:14:27.272583 27288 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:14:27.272606 27288 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:14:27.273682 27288 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:14:27.273718 27288 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:14:27.273738 27288 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260923 23:14:27.273751 27288 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260923 23:14:27.273767 27288 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.227.193
I20260923 23:14:27.275130 27288 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:34703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.193.keytab
--principal=kudu/127.24.227.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.227.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.227.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.227.254:42395
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27288
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.227.193
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:14:27.275309 27288 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:14:27.275470 27288 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:14:27.280201 27294 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:27.280227 27296 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:27.280194 27293 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:14:27.280442 27288 server_base.cc:1034] running on GCE node
I20260923 23:14:27.280668 27288 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:14:27.280905 27288 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:14:27.282075 27288 hybrid_clock.cc:648] HybridClock initialized: now 1790205267282069 us; error 35 us; skew 500 ppm
Sep 23 23:14:27 dist-test-slave-754n krb5kdc[27171](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205267, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:14:27.284523 27288 init.cc:377] Logged in from keytab as kudu/127.24.227.193@KRBTEST.COM (short username kudu)
I20260923 23:14:27.284956 27288 webserver.cc:460] Webserver started at http://127.24.227.193:46353/ using document root <none> and password file <none>
I20260923 23:14:27.285194 27288 fs_manager.cc:359] Metadata directory not provided
I20260923 23:14:27.285260 27288 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:14:27.285408 27288 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:14:27.286392 27288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/minicluster-data/ts-0/data/instance:
uuid: "b4a935543f574f50844743bce28cbab2"
format_stamp: "Formatted at 2026-09-23 23:14:27 on dist-test-slave-754n"
server_key: "c6039a4be2f96429ce21ecc906cbd035"
server_key_iv: "fc72775adb7814e448ccaaa0d729b556"
server_key_version: "encryptionkey@0"
I20260923 23:14:27.286742 27288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/minicluster-data/ts-0/wal/instance:
uuid: "b4a935543f574f50844743bce28cbab2"
format_stamp: "Formatted at 2026-09-23 23:14:27 on dist-test-slave-754n"
server_key: "c6039a4be2f96429ce21ecc906cbd035"
server_key_iv: "fc72775adb7814e448ccaaa0d729b556"
server_key_version: "encryptionkey@0"
I20260923 23:14:27.288035 27288 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260923 23:14:27.288750 27303 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:27.288923 27288 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 23:14:27.289027 27288 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/minicluster-data/ts-0/wal
uuid: "b4a935543f574f50844743bce28cbab2"
format_stamp: "Formatted at 2026-09-23 23:14:27 on dist-test-slave-754n"
server_key: "c6039a4be2f96429ce21ecc906cbd035"
server_key_iv: "fc72775adb7814e448ccaaa0d729b556"
server_key_version: "encryptionkey@0"
I20260923 23:14:27.289115 27288 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:14:27.329137 27288 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:14:27.331532 27288 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:14:27.331696 27288 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:14:27.332207 27288 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 23:14:27.332271 27288 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:27.332321 27288 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 23:14:27.332348 27288 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 23 23:14:27 dist-test-slave-754n krb5kdc[27171](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205267, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.193@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
I20260923 23:14:27.339238 27288 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.193:38023
I20260923 23:14:27.339318 27418 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.193:38023 every 8 connection(s)
I20260923 23:14:27.339599 27288 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/minicluster-data/ts-0/data/info.pb
I20260923 23:14:27.342088 27266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:27.332739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.193:39287 (local address 127.24.227.254:42395)
0923 23:14:27.332814 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:27.332817 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:27.333025 (+ 208us) server_negotiation.cc:407] Connection header received
0923 23:14:27.333492 (+ 467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:27.333493 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:27.333507 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:27.333586 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:27.334491 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:27.335344 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:27.336715 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:27.336921 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:27.337003 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:27.339399 (+ 2396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:27.339401 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:27.339401 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:27.339409 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:27.340846 (+ 1437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:27.341202 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:27.341203 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:27.341203 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:27.341256 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:27.341535 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:27.341536 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:27.341536 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:27.341634 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:27.341684 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:27.341958 (+ 274us) server_negotiation.cc:299] Negotiation successful
0923 23:14:27.342026 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260923 23:14:27.342285 27312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:27.332719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:42395 (local address 127.24.227.193:39287)
0923 23:14:27.332889 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:27.332927 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:27.333393 (+ 466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:27.333683 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:27.333687 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:27.334067 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:14:27.334357 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:27.334364 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:27.335440 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:27.335441 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:27.336603 (+ 1162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:27.336606 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:27.337022 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:27.337024 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:27.337154 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:27.337658 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:14:27.337665 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:14:27.339286 (+ 1621us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:14:27.340915 (+ 1629us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:27.340920 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:27.340922 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:27.341137 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:27.341330 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:27.341331 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:27.341331 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:27.341464 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:27.341694 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:14:27.341699 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:14:27.341836 (+ 137us) client_negotiation.cc:769] Sending connection context
0923 23:14:27.341937 (+ 101us) client_negotiation.cc:240] Negotiation successful
0923 23:14:27.342076 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":101,"threads_started":1}
I20260923 23:14:27.342783 27419 heartbeater.cc:347] Connected to a master server at 127.24.227.254:42395
I20260923 23:14:27.342860 27419 heartbeater.cc:464] Registering TS with master...
I20260923 23:14:27.343112 27419 heartbeater.cc:511] Master 127.24.227.254:42395 requested a full tablet report, sending...
I20260923 23:14:27.343711 27215 ts_manager.cc:194] Registered new tserver with Master: b4a935543f574f50844743bce28cbab2 (127.24.227.193:38023)
I20260923 23:14:27.344406 27215 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.227.193@KRBTEST.COM'} at 127.24.227.193:39287
I20260923 23:14:27.348901 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 27288
I20260923 23:14:27.348933 27425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:27.343821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:42395 (local address 127.24.227.193:48059)
0923 23:14:27.343986 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:27.343997 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:27.344058 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:27.344190 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:27.344190 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:27.344352 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:14:27.344403 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:27.344404 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:27.345065 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:27.345066 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:27.345570 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:27.345571 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:27.345945 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:27.345946 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:27.346029 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:27.346446 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:14:27.346455 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:14:27.346730 (+ 275us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:14:27.348134 (+ 1404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:27.348135 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:27.348135 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:27.348298 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:27.348448 (+ 150us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:27.348448 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:27.348448 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:27.348501 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:27.348708 (+ 207us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:14:27.348709 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:14:27.348751 (+ 42us) client_negotiation.cc:769] Sending connection context
0923 23:14:27.348803 (+ 52us) client_negotiation.cc:240] Negotiation successful
0923 23:14:27.348871 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":97,"threads_started":1}
I20260923 23:14:27.349007 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/minicluster-data/ts-0/wal/instance
I20260923 23:14:27.349050 27266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:27.343953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.193:48059 (local address 127.24.227.254:42395)
0923 23:14:27.343997 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:27.343999 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:27.344043 (+ 44us) server_negotiation.cc:407] Connection header received
0923 23:14:27.344121 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:27.344122 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:27.344130 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:27.344185 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:27.344485 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:27.344970 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:27.345651 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:27.345860 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:27.345944 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:27.346838 (+ 894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:27.346839 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:27.346840 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:27.346846 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:27.348068 (+ 1222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:27.348362 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:27.348363 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:27.348363 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:27.348409 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:27.348549 (+ 140us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:27.348550 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:27.348550 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:27.348638 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:27.348686 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:27.348850 (+ 164us) server_negotiation.cc:299] Negotiation successful
0923 23:14:27.348929 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260923 23:14:27.349192 25487 external_mini_cluster.cc:1428] Setting key ec29b061c8d34e03e40bc6e32ce1fa1f
WARNING: no policy specified for kudu/127.24.227.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.227.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.227.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.194.keytab.
Entry for principal kudu/127.24.227.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.194.keytab.
I20260923 23:14:27.372294 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.227.194:0
--local_ip_for_outbound_sockets=127.24.227.194
--webserver_interface=127.24.227.194
--webserver_port=0
--tserver_master_addrs=127.24.227.254:42395
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.194.keytab
--principal=kudu/127.24.227.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:34703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:14:27.449322 27428 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:14:27.449530 27428 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:14:27.449568 27428 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:14:27.450659 27428 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:14:27.450721 27428 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:14:27.450753 27428 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260923 23:14:27.450779 27428 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260923 23:14:27.450809 27428 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.227.194
I20260923 23:14:27.452309 27428 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:34703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.194.keytab
--principal=kudu/127.24.227.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.227.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.227.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.227.254:42395
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27428
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.227.194
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:14:27.452545 27428 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:14:27.452761 27428 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:14:27.457221 27434 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:27.457211 27433 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:27.457212 27436 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:14:27.457281 27428 server_base.cc:1034] running on GCE node
I20260923 23:14:27.457563 27428 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:14:27.457808 27428 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:14:27.458977 27428 hybrid_clock.cc:648] HybridClock initialized: now 1790205267458972 us; error 33 us; skew 500 ppm
Sep 23 23:14:27 dist-test-slave-754n krb5kdc[27171](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205267, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:14:27.461319 27428 init.cc:377] Logged in from keytab as kudu/127.24.227.194@KRBTEST.COM (short username kudu)
I20260923 23:14:27.461725 27428 webserver.cc:460] Webserver started at http://127.24.227.194:43983/ using document root <none> and password file <none>
I20260923 23:14:27.461882 27428 fs_manager.cc:359] Metadata directory not provided
I20260923 23:14:27.461920 27428 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:14:27.462000 27428 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:14:27.462997 27428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/minicluster-data/ts-1/data/instance:
uuid: "21a48e797c1942b4bfa349ac15116147"
format_stamp: "Formatted at 2026-09-23 23:14:27 on dist-test-slave-754n"
server_key: "e7a6f7328571ebe031520d7da18e822a"
server_key_iv: "57d1062275eed525a4d788f57fadd976"
server_key_version: "encryptionkey@0"
I20260923 23:14:27.463282 27428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/minicluster-data/ts-1/wal/instance:
uuid: "21a48e797c1942b4bfa349ac15116147"
format_stamp: "Formatted at 2026-09-23 23:14:27 on dist-test-slave-754n"
server_key: "e7a6f7328571ebe031520d7da18e822a"
server_key_iv: "57d1062275eed525a4d788f57fadd976"
server_key_version: "encryptionkey@0"
I20260923 23:14:27.464622 27428 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 23:14:27.465365 27443 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:27.465580 27428 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 23:14:27.465665 27428 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/minicluster-data/ts-1/wal
uuid: "21a48e797c1942b4bfa349ac15116147"
format_stamp: "Formatted at 2026-09-23 23:14:27 on dist-test-slave-754n"
server_key: "e7a6f7328571ebe031520d7da18e822a"
server_key_iv: "57d1062275eed525a4d788f57fadd976"
server_key_version: "encryptionkey@0"
I20260923 23:14:27.465737 27428 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:14:27.483971 27428 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:14:27.486323 27428 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:14:27.486476 27428 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:14:27.486853 27428 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 23:14:27.486889 27428 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:27.486944 27428 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 23:14:27.486970 27428 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:27.493232 27428 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.194:45731
I20260923 23:14:27.493305 27558 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.194:45731 every 8 connection(s)
I20260923 23:14:27.493566 27428 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/minicluster-data/ts-1/data/info.pb
Sep 23 23:14:27 dist-test-slave-754n krb5kdc[27171](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205267, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.194@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
I20260923 23:14:27.496416 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 27428
I20260923 23:14:27.496513 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/minicluster-data/ts-1/wal/instance
I20260923 23:14:27.496706 25487 external_mini_cluster.cc:1428] Setting key cd8cdd18af5bc1ca1b7827578ba4a800
I20260923 23:14:27.496893 27266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:27.487293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.194:53161 (local address 127.24.227.254:42395)
0923 23:14:27.487335 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:27.487337 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:27.487545 (+ 208us) server_negotiation.cc:407] Connection header received
0923 23:14:27.488068 (+ 523us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:27.488070 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:27.488083 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:27.488142 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:27.488951 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:27.489826 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:27.491428 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:27.491699 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:27.491785 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:27.494159 (+ 2374us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:27.494160 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:27.494161 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:27.494167 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:27.495532 (+ 1365us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:27.495909 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:27.495910 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:27.495911 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:27.495973 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:27.496181 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:27.496182 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:27.496182 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:27.496296 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:27.496344 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:27.496690 (+ 346us) server_negotiation.cc:299] Negotiation successful
0923 23:14:27.496787 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260923 23:14:27.497071 27452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:27.487278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:42395 (local address 127.24.227.194:53161)
0923 23:14:27.487431 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:27.487460 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:27.487966 (+ 506us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:27.488206 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:27.488210 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:27.488569 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:14:27.488831 (+ 262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:27.488838 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:27.489960 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:27.489961 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:27.491348 (+ 1387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:27.491351 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:27.491795 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:27.491796 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:27.491948 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:27.492434 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:14:27.492444 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:14:27.494074 (+ 1630us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:14:27.495600 (+ 1526us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:27.495604 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:27.495606 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:27.495812 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:27.496036 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:27.496036 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:27.496036 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:27.496125 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:27.496360 (+ 235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:14:27.496367 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:14:27.496545 (+ 178us) client_negotiation.cc:769] Sending connection context
0923 23:14:27.496670 (+ 125us) client_negotiation.cc:240] Negotiation successful
0923 23:14:27.496829 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":90,"threads_started":1}
I20260923 23:14:27.497605 27559 heartbeater.cc:347] Connected to a master server at 127.24.227.254:42395
I20260923 23:14:27.497684 27559 heartbeater.cc:464] Registering TS with master...
I20260923 23:14:27.497864 27559 heartbeater.cc:511] Master 127.24.227.254:42395 requested a full tablet report, sending...
I20260923 23:14:27.498198 27215 ts_manager.cc:194] Registered new tserver with Master: 21a48e797c1942b4bfa349ac15116147 (127.24.227.194:45731)
I20260923 23:14:27.498679 27215 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.227.194@KRBTEST.COM'} at 127.24.227.194:53161
I20260923 23:14:27.504405 27266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:27.498632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.194:38857 (local address 127.24.227.254:42395)
0923 23:14:27.498664 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:27.498666 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:27.498770 (+ 104us) server_negotiation.cc:407] Connection header received
0923 23:14:27.499072 (+ 302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:27.499073 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:27.499081 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:27.499172 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:27.499479 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:27.500021 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:27.500704 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:27.500898 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:27.500968 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:27.501888 (+ 920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:27.501889 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:27.501889 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:27.501895 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:27.503193 (+ 1298us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:27.503595 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:27.503596 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:27.503596 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:27.503647 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:27.503854 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:27.503855 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:27.503855 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:27.504027 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:27.504107 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:27.504249 (+ 142us) server_negotiation.cc:299] Negotiation successful
0923 23:14:27.504317 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260923 23:14:27.504415 27565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:27.498589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:42395 (local address 127.24.227.194:38857)
0923 23:14:27.498703 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:27.498711 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:27.498995 (+ 284us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:27.499178 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:27.499179 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:27.499356 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:14:27.499401 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:27.499403 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:27.500111 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:27.500112 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:27.500612 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:27.500613 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:27.500967 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:27.500968 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:27.501041 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:27.501494 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:14:27.501507 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:14:27.501776 (+ 269us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:14:27.503289 (+ 1513us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:27.503290 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:27.503290 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:27.503538 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:27.503732 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:27.503733 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:27.503733 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:27.503788 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:27.504099 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:14:27.504099 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:14:27.504142 (+ 43us) client_negotiation.cc:769] Sending connection context
0923 23:14:27.504226 (+ 84us) client_negotiation.cc:240] Negotiation successful
0923 23:14:27.504319 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":59,"threads_started":1}
WARNING: no policy specified for kudu/127.24.227.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.227.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.227.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.195.keytab.
Entry for principal kudu/127.24.227.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.195.keytab.
I20260923 23:14:27.520416 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.227.195:0
--local_ip_for_outbound_sockets=127.24.227.195
--webserver_interface=127.24.227.195
--webserver_port=0
--tserver_master_addrs=127.24.227.254:42395
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.195.keytab
--principal=kudu/127.24.227.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:34703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:14:27.598472 27568 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:14:27.598639 27568 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:14:27.598661 27568 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:14:27.599795 27568 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:14:27.599836 27568 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:14:27.599856 27568 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260923 23:14:27.599928 27568 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260923 23:14:27.599972 27568 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.227.195
I20260923 23:14:27.601385 27568 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:34703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.195.keytab
--principal=kudu/127.24.227.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.227.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.227.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.227.254:42395
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27568
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.227.195
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:14:27.601601 27568 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:14:27.601809 27568 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:14:27.606470 27576 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:14:27.606554 27568 server_base.cc:1034] running on GCE node
W20260923 23:14:27.606462 27573 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:27.606462 27574 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:14:27.606856 27568 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:14:27.607124 27568 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:14:27.608289 27568 hybrid_clock.cc:648] HybridClock initialized: now 1790205267608272 us; error 45 us; skew 500 ppm
Sep 23 23:14:27 dist-test-slave-754n krb5kdc[27171](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205267, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:14:27.615931 27568 init.cc:377] Logged in from keytab as kudu/127.24.227.195@KRBTEST.COM (short username kudu)
I20260923 23:14:27.616319 27568 webserver.cc:460] Webserver started at http://127.24.227.195:44421/ using document root <none> and password file <none>
I20260923 23:14:27.616547 27568 fs_manager.cc:359] Metadata directory not provided
I20260923 23:14:27.616616 27568 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:14:27.616760 27568 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:14:27.617699 27568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/minicluster-data/ts-2/data/instance:
uuid: "78106638c5c545219d95183f82117c82"
format_stamp: "Formatted at 2026-09-23 23:14:27 on dist-test-slave-754n"
server_key: "b19ed4f87dbdf936144965bd9b7df126"
server_key_iv: "b91de39251a79959d8edb3b2242caf31"
server_key_version: "encryptionkey@0"
I20260923 23:14:27.618095 27568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/minicluster-data/ts-2/wal/instance:
uuid: "78106638c5c545219d95183f82117c82"
format_stamp: "Formatted at 2026-09-23 23:14:27 on dist-test-slave-754n"
server_key: "b19ed4f87dbdf936144965bd9b7df126"
server_key_iv: "b91de39251a79959d8edb3b2242caf31"
server_key_version: "encryptionkey@0"
I20260923 23:14:27.619339 27568 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 23:14:27.620057 27583 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:27.620220 27568 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:27.620314 27568 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/minicluster-data/ts-2/wal
uuid: "78106638c5c545219d95183f82117c82"
format_stamp: "Formatted at 2026-09-23 23:14:27 on dist-test-slave-754n"
server_key: "b19ed4f87dbdf936144965bd9b7df126"
server_key_iv: "b91de39251a79959d8edb3b2242caf31"
server_key_version: "encryptionkey@0"
I20260923 23:14:27.620394 27568 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:14:27.633795 27568 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:14:27.635955 27568 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:14:27.636109 27568 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:14:27.636534 27568 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 23:14:27.636620 27568 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:27.636660 27568 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 23:14:27.636689 27568 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:27.643390 27568 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.195:38423
Sep 23 23:14:27 dist-test-slave-754n krb5kdc[27171](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205267, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.195@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
I20260923 23:14:27.643770 27568 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/minicluster-data/ts-2/data/info.pb
I20260923 23:14:27.644924 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 27568
I20260923 23:14:27.645002 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/minicluster-data/ts-2/wal/instance
I20260923 23:14:27.645219 25487 external_mini_cluster.cc:1428] Setting key 9bb4fed25797d31c3e634f97b157db0c
I20260923 23:14:27.645434 27698 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.195:38423 every 8 connection(s)
I20260923 23:14:27.647851 27266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:27.637036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.195:32869 (local address 127.24.227.254:42395)
0923 23:14:27.637102 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:27.637104 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:27.637295 (+ 191us) server_negotiation.cc:407] Connection header received
0923 23:14:27.637746 (+ 451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:27.637747 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:27.637759 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:27.637821 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:27.638613 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:27.639474 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:27.640988 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:27.641342 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:27.641417 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:27.644166 (+ 2749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:27.644168 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:27.644168 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:27.644177 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:27.646500 (+ 2323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:27.646902 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:27.646903 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:27.646903 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:27.646971 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:27.647223 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:27.647224 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:27.647224 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:27.647339 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:27.647411 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:27.647681 (+ 270us) server_negotiation.cc:299] Negotiation successful
0923 23:14:27.647766 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260923 23:14:27.648048 27592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:27.637034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:42395 (local address 127.24.227.195:32869)
0923 23:14:27.637190 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:27.637219 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:27.637647 (+ 428us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:27.637890 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:27.637893 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:27.638240 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:14:27.638496 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:27.638502 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:27.639573 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:27.639574 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:27.640892 (+ 1318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:27.640896 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:27.641452 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:27.641453 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:27.641545 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:27.642036 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:14:27.642051 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:14:27.644057 (+ 2006us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:14:27.646597 (+ 2540us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:27.646602 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:27.646604 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:27.646815 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:27.647036 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:27.647037 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:27.647037 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:27.647149 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:27.647413 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:14:27.647419 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:14:27.647545 (+ 126us) client_negotiation.cc:769] Sending connection context
0923 23:14:27.647644 (+ 99us) client_negotiation.cc:240] Negotiation successful
0923 23:14:27.647788 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":85,"threads_started":1}
I20260923 23:14:27.648494 27699 heartbeater.cc:347] Connected to a master server at 127.24.227.254:42395
I20260923 23:14:27.648574 27699 heartbeater.cc:464] Registering TS with master...
I20260923 23:14:27.648737 27699 heartbeater.cc:511] Master 127.24.227.254:42395 requested a full tablet report, sending...
I20260923 23:14:27.649067 27215 ts_manager.cc:194] Registered new tserver with Master: 78106638c5c545219d95183f82117c82 (127.24.227.195:38423)
I20260923 23:14:27.649560 27215 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.227.195@KRBTEST.COM'} at 127.24.227.195:32869
I20260923 23:14:27.657506 25487 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 23:14:27.658771 27266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:27.649887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.195:48905 (local address 127.24.227.254:42395)
0923 23:14:27.650004 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:27.650005 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:27.650199 (+ 194us) server_negotiation.cc:407] Connection header received
0923 23:14:27.650269 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:27.650270 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:27.650281 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:27.650346 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:27.650731 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:27.651685 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:27.652681 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:27.652954 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:27.653036 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:27.655493 (+ 2457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:27.655494 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:27.655494 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:27.655500 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:27.657416 (+ 1916us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:27.657806 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:27.657807 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:27.657808 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:27.657865 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:27.658096 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:27.658097 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:27.658097 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:27.658339 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:27.658408 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:27.658644 (+ 236us) server_negotiation.cc:299] Negotiation successful
0923 23:14:27.658705 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260923 23:14:27.659116 27705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:27.649799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:42395 (local address 127.24.227.195:48905)
0923 23:14:27.650144 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:27.650155 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:27.650211 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:27.650364 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:27.650365 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:27.650583 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:14:27.650666 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:27.650668 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:27.651812 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:27.651813 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:27.652593 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:27.652596 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:27.653038 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:27.653065 (+ 27us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:27.653141 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:27.653566 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:14:27.653571 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:14:27.653979 (+ 408us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:14:27.657492 (+ 3513us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:27.657493 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:27.657493 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:27.657719 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:27.657940 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:27.657941 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:27.657942 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:27.658016 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:27.658450 (+ 434us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:14:27.658451 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:14:27.658508 (+ 57us) client_negotiation.cc:769] Sending connection context
0923 23:14:27.658579 (+ 71us) client_negotiation.cc:240] Negotiation successful
0923 23:14:27.658699 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":59,"threads_started":1}
Sep 23 23:14:27 dist-test-slave-754n krb5kdc[27171](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205267, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 23 23:14:27 dist-test-slave-754n krb5kdc[27171](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205267, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
I20260923 23:14:27.687060 27266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:27.677860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50660 (local address 127.24.227.254:42395)
0923 23:14:27.677909 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:27.677911 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:27.677920 (+ 9us) server_negotiation.cc:407] Connection header received
0923 23:14:27.677998 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:27.677998 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:27.678009 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:27.678067 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:27.678796 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:27.679694 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:27.680789 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:27.681124 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:27.681201 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:27.684087 (+ 2886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:27.684089 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:27.684089 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:27.684096 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:27.685886 (+ 1790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:27.686258 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:27.686259 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:27.686259 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:27.686320 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:27.686526 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:27.686527 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:27.686527 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:27.686665 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:27.686774 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:27.686865 (+ 91us) server_negotiation.cc:299] Negotiation successful
0923 23:14:27.686941 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260923 23:14:27.688839 27215 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50660:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 23:14:27.690354 27215 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 23:14:27.696385 27717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:27.692718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.194:45731 (local address 127.0.0.1:43922)
0923 23:14:27.692977 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:27.692986 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:27.693081 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:27.693461 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:27.693462 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:27.693465 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:14:27.693522 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:27.693524 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:27.694432 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:27.694434 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:27.695232 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:27.695234 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:27.696129 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:27.696130 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:27.696199 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:27.696213 (+ 14us) client_negotiation.cc:769] Sending connection context
0923 23:14:27.696281 (+ 68us) client_negotiation.cc:240] Negotiation successful
0923 23:14:27.696322 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":75,"threads_started":1}
I20260923 23:14:27.696425 27721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:27.693131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43922 (local address 127.24.227.194:45731)
0923 23:14:27.693267 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:27.693272 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:27.693281 (+ 9us) server_negotiation.cc:407] Connection header received
0923 23:14:27.693316 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:27.693320 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:27.693358 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:27.693603 (+ 245us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:14:27.693666 (+ 63us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:27.694336 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:27.695376 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:27.696051 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:27.696123 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:27.696173 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:27.696304 (+ 131us) server_negotiation.cc:299] Negotiation successful
0923 23:14:27.696343 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":75,"threads_started":1}
I20260923 23:14:27.697079 27718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:27.692724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.193:38023 (local address 127.0.0.1:41908)
0923 23:14:27.693267 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:27.693276 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:27.693346 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:27.693617 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:27.693618 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:27.693620 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:14:27.693658 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:27.693660 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:27.694850 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:27.694851 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:27.695932 (+ 1081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:27.695933 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:27.696827 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:27.696828 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:27.696889 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:27.696893 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 23:14:27.696974 (+ 81us) client_negotiation.cc:240] Negotiation successful
0923 23:14:27.697012 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":76,"threads_started":1}
I20260923 23:14:27.697129 27719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:27.692809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41908 (local address 127.24.227.193:38023)
0923 23:14:27.693220 (+ 411us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:27.693224 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:27.693319 (+ 95us) server_negotiation.cc:407] Connection header received
0923 23:14:27.693435 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:27.693437 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:27.693454 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:27.693515 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:14:27.693734 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:27.694768 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:27.696028 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:27.696734 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:27.696801 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:27.696868 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:27.696990 (+ 122us) server_negotiation.cc:299] Negotiation successful
0923 23:14:27.697025 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":84,"threads_started":1}
I20260923 23:14:27.697165 27716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:27.692718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.195:38423 (local address 127.0.0.1:51296)
0923 23:14:27.692944 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:27.692985 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:27.693077 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:27.693346 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:27.693349 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:27.693351 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:14:27.693414 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:27.693415 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:27.694710 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:27.694711 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:27.695853 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:27.695855 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:27.696930 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:27.696931 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:27.696995 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:27.697000 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 23:14:27.697044 (+ 44us) client_negotiation.cc:240] Negotiation successful
0923 23:14:27.697095 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":49,"threads_started":1}
I20260923 23:14:27.697165 27493 tablet_service.cc:1467] Processing CreateTablet for tablet 139ebcfbfa8e4f948d217fdce2642b8f (DEFAULT_TABLE table=test-table [id=3e83e0fac380470cbbbf6202ea910830]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 23:14:27.697510 27720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:27.692927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51296 (local address 127.24.227.195:38423)
0923 23:14:27.693124 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:27.693129 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:27.693138 (+ 9us) server_negotiation.cc:407] Connection header received
0923 23:14:27.693179 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:27.693181 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:27.693197 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:27.693271 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:14:27.693501 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:27.694619 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:27.696004 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:27.696772 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:27.696851 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:27.696898 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:27.697319 (+ 421us) server_negotiation.cc:299] Negotiation successful
0923 23:14:27.697370 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":99,"threads_started":1}
I20260923 23:14:27.697583 27493 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 139ebcfbfa8e4f948d217fdce2642b8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:14:27.697774 27353 tablet_service.cc:1467] Processing CreateTablet for tablet 139ebcfbfa8e4f948d217fdce2642b8f (DEFAULT_TABLE table=test-table [id=3e83e0fac380470cbbbf6202ea910830]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 23:14:27.698041 27353 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 139ebcfbfa8e4f948d217fdce2642b8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:14:27.698035 27633 tablet_service.cc:1467] Processing CreateTablet for tablet 139ebcfbfa8e4f948d217fdce2642b8f (DEFAULT_TABLE table=test-table [id=3e83e0fac380470cbbbf6202ea910830]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 23:14:27.698344 27633 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 139ebcfbfa8e4f948d217fdce2642b8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:14:27.700975 27722 tablet_bootstrap.cc:492] T 139ebcfbfa8e4f948d217fdce2642b8f P b4a935543f574f50844743bce28cbab2: Bootstrap starting.
I20260923 23:14:27.701246 27723 tablet_bootstrap.cc:492] T 139ebcfbfa8e4f948d217fdce2642b8f P 21a48e797c1942b4bfa349ac15116147: Bootstrap starting.
I20260923 23:14:27.701622 27722 tablet_bootstrap.cc:654] T 139ebcfbfa8e4f948d217fdce2642b8f P b4a935543f574f50844743bce28cbab2: Neither blocks nor log segments found. Creating new log.
I20260923 23:14:27.701819 27723 tablet_bootstrap.cc:654] T 139ebcfbfa8e4f948d217fdce2642b8f P 21a48e797c1942b4bfa349ac15116147: Neither blocks nor log segments found. Creating new log.
I20260923 23:14:27.701974 27722 log.cc:824] T 139ebcfbfa8e4f948d217fdce2642b8f P b4a935543f574f50844743bce28cbab2: Log is configured to *not* fsync() on all Append() calls
I20260923 23:14:27.702010 27724 tablet_bootstrap.cc:492] T 139ebcfbfa8e4f948d217fdce2642b8f P 78106638c5c545219d95183f82117c82: Bootstrap starting.
I20260923 23:14:27.702148 27723 log.cc:824] T 139ebcfbfa8e4f948d217fdce2642b8f P 21a48e797c1942b4bfa349ac15116147: Log is configured to *not* fsync() on all Append() calls
I20260923 23:14:27.702801 27722 tablet_bootstrap.cc:492] T 139ebcfbfa8e4f948d217fdce2642b8f P b4a935543f574f50844743bce28cbab2: No bootstrap required, opened a new log
I20260923 23:14:27.702855 27722 ts_tablet_manager.cc:1397] T 139ebcfbfa8e4f948d217fdce2642b8f P b4a935543f574f50844743bce28cbab2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 23:14:27.702942 27724 tablet_bootstrap.cc:654] T 139ebcfbfa8e4f948d217fdce2642b8f P 78106638c5c545219d95183f82117c82: Neither blocks nor log segments found. Creating new log.
I20260923 23:14:27.702961 27723 tablet_bootstrap.cc:492] T 139ebcfbfa8e4f948d217fdce2642b8f P 21a48e797c1942b4bfa349ac15116147: No bootstrap required, opened a new log
I20260923 23:14:27.703254 27723 ts_tablet_manager.cc:1397] T 139ebcfbfa8e4f948d217fdce2642b8f P 21a48e797c1942b4bfa349ac15116147: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260923 23:14:27.703264 27724 log.cc:824] T 139ebcfbfa8e4f948d217fdce2642b8f P 78106638c5c545219d95183f82117c82: Log is configured to *not* fsync() on all Append() calls
I20260923 23:14:27.704221 27724 tablet_bootstrap.cc:492] T 139ebcfbfa8e4f948d217fdce2642b8f P 78106638c5c545219d95183f82117c82: No bootstrap required, opened a new log
I20260923 23:14:27.704288 27724 ts_tablet_manager.cc:1397] T 139ebcfbfa8e4f948d217fdce2642b8f P 78106638c5c545219d95183f82117c82: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260923 23:14:27.704593 27722 raft_consensus.cc:348] T 139ebcfbfa8e4f948d217fdce2642b8f P b4a935543f574f50844743bce28cbab2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21a48e797c1942b4bfa349ac15116147" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 45731 } } peers { permanent_uuid: "78106638c5c545219d95183f82117c82" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 38423 } } peers { permanent_uuid: "b4a935543f574f50844743bce28cbab2" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 38023 } }
I20260923 23:14:27.704712 27722 raft_consensus.cc:374] T 139ebcfbfa8e4f948d217fdce2642b8f P b4a935543f574f50844743bce28cbab2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:14:27.704733 27722 raft_consensus.cc:729] T 139ebcfbfa8e4f948d217fdce2642b8f P b4a935543f574f50844743bce28cbab2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4a935543f574f50844743bce28cbab2, State: Initialized, Role: FOLLOWER
I20260923 23:14:27.704849 27722 consensus_queue.cc:260] T 139ebcfbfa8e4f948d217fdce2642b8f P b4a935543f574f50844743bce28cbab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21a48e797c1942b4bfa349ac15116147" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 45731 } } peers { permanent_uuid: "78106638c5c545219d95183f82117c82" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 38423 } } peers { permanent_uuid: "b4a935543f574f50844743bce28cbab2" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 38023 } }
I20260923 23:14:27.705071 27722 ts_tablet_manager.cc:1428] T 139ebcfbfa8e4f948d217fdce2642b8f P b4a935543f574f50844743bce28cbab2: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260923 23:14:27.705087 27419 heartbeater.cc:503] Master 127.24.227.254:42395 was elected leader, sending a full tablet report...
I20260923 23:14:27.705657 27723 raft_consensus.cc:348] T 139ebcfbfa8e4f948d217fdce2642b8f P 21a48e797c1942b4bfa349ac15116147 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21a48e797c1942b4bfa349ac15116147" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 45731 } } peers { permanent_uuid: "78106638c5c545219d95183f82117c82" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 38423 } } peers { permanent_uuid: "b4a935543f574f50844743bce28cbab2" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 38023 } }
I20260923 23:14:27.705802 27723 raft_consensus.cc:374] T 139ebcfbfa8e4f948d217fdce2642b8f P 21a48e797c1942b4bfa349ac15116147 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:14:27.705857 27723 raft_consensus.cc:729] T 139ebcfbfa8e4f948d217fdce2642b8f P 21a48e797c1942b4bfa349ac15116147 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21a48e797c1942b4bfa349ac15116147, State: Initialized, Role: FOLLOWER
I20260923 23:14:27.705963 27723 consensus_queue.cc:260] T 139ebcfbfa8e4f948d217fdce2642b8f P 21a48e797c1942b4bfa349ac15116147 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21a48e797c1942b4bfa349ac15116147" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 45731 } } peers { permanent_uuid: "78106638c5c545219d95183f82117c82" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 38423 } } peers { permanent_uuid: "b4a935543f574f50844743bce28cbab2" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 38023 } }
I20260923 23:14:27.706210 27723 ts_tablet_manager.cc:1428] T 139ebcfbfa8e4f948d217fdce2642b8f P 21a48e797c1942b4bfa349ac15116147: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260923 23:14:27.706264 27559 heartbeater.cc:503] Master 127.24.227.254:42395 was elected leader, sending a full tablet report...
I20260923 23:14:27.706230 27724 raft_consensus.cc:348] T 139ebcfbfa8e4f948d217fdce2642b8f P 78106638c5c545219d95183f82117c82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21a48e797c1942b4bfa349ac15116147" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 45731 } } peers { permanent_uuid: "78106638c5c545219d95183f82117c82" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 38423 } } peers { permanent_uuid: "b4a935543f574f50844743bce28cbab2" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 38023 } }
I20260923 23:14:27.706364 27724 raft_consensus.cc:374] T 139ebcfbfa8e4f948d217fdce2642b8f P 78106638c5c545219d95183f82117c82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:14:27.706400 27724 raft_consensus.cc:729] T 139ebcfbfa8e4f948d217fdce2642b8f P 78106638c5c545219d95183f82117c82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78106638c5c545219d95183f82117c82, State: Initialized, Role: FOLLOWER
I20260923 23:14:27.706498 27724 consensus_queue.cc:260] T 139ebcfbfa8e4f948d217fdce2642b8f P 78106638c5c545219d95183f82117c82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21a48e797c1942b4bfa349ac15116147" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 45731 } } peers { permanent_uuid: "78106638c5c545219d95183f82117c82" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 38423 } } peers { permanent_uuid: "b4a935543f574f50844743bce28cbab2" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 38023 } }
I20260923 23:14:27.706727 27724 ts_tablet_manager.cc:1428] T 139ebcfbfa8e4f948d217fdce2642b8f P 78106638c5c545219d95183f82117c82: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260923 23:14:27.706820 27699 heartbeater.cc:503] Master 127.24.227.254:42395 was elected leader, sending a full tablet report...
I20260923 23:14:27.722445 27729 raft_consensus.cc:482] T 139ebcfbfa8e4f948d217fdce2642b8f P 21a48e797c1942b4bfa349ac15116147 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 23:14:27.722546 27729 raft_consensus.cc:504] T 139ebcfbfa8e4f948d217fdce2642b8f P 21a48e797c1942b4bfa349ac15116147 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21a48e797c1942b4bfa349ac15116147" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 45731 } } peers { permanent_uuid: "78106638c5c545219d95183f82117c82" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 38423 } } peers { permanent_uuid: "b4a935543f574f50844743bce28cbab2" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 38023 } }
I20260923 23:14:27.722846 27729 leader_election.cc:288] T 139ebcfbfa8e4f948d217fdce2642b8f P 21a48e797c1942b4bfa349ac15116147 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 78106638c5c545219d95183f82117c82 (127.24.227.195:38423), b4a935543f574f50844743bce28cbab2 (127.24.227.193:38023)
I20260923 23:14:27.726446 27719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:27.723063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.194:38227 (local address 127.24.227.193:38023)
0923 23:14:27.723116 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:27.723118 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:27.723127 (+ 9us) server_negotiation.cc:407] Connection header received
0923 23:14:27.723183 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:27.723184 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:27.723193 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:27.723245 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:14:27.723437 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:27.724230 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:27.725565 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:27.726104 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:27.726179 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:27.726186 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:27.726352 (+ 166us) server_negotiation.cc:299] Negotiation successful
0923 23:14:27.726385 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260923 23:14:27.726446 27720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:27.723121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.194:44815 (local address 127.24.227.195:38423)
0923 23:14:27.723156 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:27.723158 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:27.723425 (+ 267us) server_negotiation.cc:407] Connection header received
0923 23:14:27.723454 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:27.723455 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:27.723463 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:27.723516 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:14:27.723679 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:27.724369 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:27.725559 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:27.726104 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:27.726179 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:27.726186 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:27.726352 (+ 166us) server_negotiation.cc:299] Negotiation successful
0923 23:14:27.726385 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260923 23:14:27.726480 27452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:27.723017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.193:38023 (local address 127.24.227.194:38227)
0923 23:14:27.723054 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:27.723079 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:27.723121 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:27.723291 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:27.723293 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:27.723294 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:14:27.723364 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:27.723366 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:27.724309 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:27.724310 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:27.725455 (+ 1145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:27.725458 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:27.726186 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:27.726186 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:27.726272 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:27.726276 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 23:14:27.726332 (+ 56us) client_negotiation.cc:240] Negotiation successful
0923 23:14:27.726389 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22,"mutex_wait_us":9}
I20260923 23:14:27.726490 27731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:27.723018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.195:38423 (local address 127.24.227.194:44815)
0923 23:14:27.723341 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:27.723352 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:27.723412 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:27.723518 (+ 106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:27.723519 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:27.723520 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:14:27.723591 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:27.723592 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:27.724438 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:27.724439 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:27.725487 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:27.725488 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:27.726186 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:27.726186 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:27.726272 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:27.726276 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 23:14:27.726332 (+ 56us) client_negotiation.cc:240] Negotiation successful
0923 23:14:27.726389 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"mutex_wait_us":181,"thread_start_us":91,"threads_started":1}
I20260923 23:14:27.726859 27373 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "139ebcfbfa8e4f948d217fdce2642b8f" candidate_uuid: "21a48e797c1942b4bfa349ac15116147" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4a935543f574f50844743bce28cbab2" is_pre_election: true
I20260923 23:14:27.726904 27653 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "139ebcfbfa8e4f948d217fdce2642b8f" candidate_uuid: "21a48e797c1942b4bfa349ac15116147" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78106638c5c545219d95183f82117c82" is_pre_election: true
I20260923 23:14:27.726984 27373 raft_consensus.cc:2454] T 139ebcfbfa8e4f948d217fdce2642b8f P b4a935543f574f50844743bce28cbab2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 21a48e797c1942b4bfa349ac15116147 in term 0.
I20260923 23:14:27.727006 27653 raft_consensus.cc:2454] T 139ebcfbfa8e4f948d217fdce2642b8f P 78106638c5c545219d95183f82117c82 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 21a48e797c1942b4bfa349ac15116147 in term 0.
I20260923 23:14:27.727203 27444 leader_election.cc:302] T 139ebcfbfa8e4f948d217fdce2642b8f P 21a48e797c1942b4bfa349ac15116147 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21a48e797c1942b4bfa349ac15116147, b4a935543f574f50844743bce28cbab2; no voters:
I20260923 23:14:27.727341 27729 raft_consensus.cc:2781] T 139ebcfbfa8e4f948d217fdce2642b8f P 21a48e797c1942b4bfa349ac15116147 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 23:14:27.727406 27729 raft_consensus.cc:482] T 139ebcfbfa8e4f948d217fdce2642b8f P 21a48e797c1942b4bfa349ac15116147 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 23:14:27.727428 27729 raft_consensus.cc:3037] T 139ebcfbfa8e4f948d217fdce2642b8f P 21a48e797c1942b4bfa349ac15116147 [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:14:27.728195 27729 raft_consensus.cc:504] T 139ebcfbfa8e4f948d217fdce2642b8f P 21a48e797c1942b4bfa349ac15116147 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21a48e797c1942b4bfa349ac15116147" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 45731 } } peers { permanent_uuid: "78106638c5c545219d95183f82117c82" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 38423 } } peers { permanent_uuid: "b4a935543f574f50844743bce28cbab2" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 38023 } }
I20260923 23:14:27.728327 27729 leader_election.cc:288] T 139ebcfbfa8e4f948d217fdce2642b8f P 21a48e797c1942b4bfa349ac15116147 [CANDIDATE]: Term 1 election: Requested vote from peers 78106638c5c545219d95183f82117c82 (127.24.227.195:38423), b4a935543f574f50844743bce28cbab2 (127.24.227.193:38023)
I20260923 23:14:27.728449 27653 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "139ebcfbfa8e4f948d217fdce2642b8f" candidate_uuid: "21a48e797c1942b4bfa349ac15116147" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78106638c5c545219d95183f82117c82"
I20260923 23:14:27.728502 27373 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "139ebcfbfa8e4f948d217fdce2642b8f" candidate_uuid: "21a48e797c1942b4bfa349ac15116147" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4a935543f574f50844743bce28cbab2"
I20260923 23:14:27.728521 27653 raft_consensus.cc:3037] T 139ebcfbfa8e4f948d217fdce2642b8f P 78106638c5c545219d95183f82117c82 [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:14:27.728574 27373 raft_consensus.cc:3037] T 139ebcfbfa8e4f948d217fdce2642b8f P b4a935543f574f50844743bce28cbab2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:14:27.729305 27653 raft_consensus.cc:2454] T 139ebcfbfa8e4f948d217fdce2642b8f P 78106638c5c545219d95183f82117c82 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21a48e797c1942b4bfa349ac15116147 in term 1.
I20260923 23:14:27.729342 27373 raft_consensus.cc:2454] T 139ebcfbfa8e4f948d217fdce2642b8f P b4a935543f574f50844743bce28cbab2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21a48e797c1942b4bfa349ac15116147 in term 1.
I20260923 23:14:27.729738 27446 leader_election.cc:302] T 139ebcfbfa8e4f948d217fdce2642b8f P 21a48e797c1942b4bfa349ac15116147 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21a48e797c1942b4bfa349ac15116147, 78106638c5c545219d95183f82117c82; no voters:
I20260923 23:14:27.729897 27729 raft_consensus.cc:2781] T 139ebcfbfa8e4f948d217fdce2642b8f P 21a48e797c1942b4bfa349ac15116147 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 23:14:27.730018 27729 raft_consensus.cc:686] T 139ebcfbfa8e4f948d217fdce2642b8f P 21a48e797c1942b4bfa349ac15116147 [term 1 LEADER]: Becoming Leader. State: Replica: 21a48e797c1942b4bfa349ac15116147, State: Running, Role: LEADER
I20260923 23:14:27.730115 27729 consensus_queue.cc:237] T 139ebcfbfa8e4f948d217fdce2642b8f P 21a48e797c1942b4bfa349ac15116147 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21a48e797c1942b4bfa349ac15116147" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 45731 } } peers { permanent_uuid: "78106638c5c545219d95183f82117c82" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 38423 } } peers { permanent_uuid: "b4a935543f574f50844743bce28cbab2" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 38023 } }
I20260923 23:14:27.730831 27215 catalog_manager.cc:5495] T 139ebcfbfa8e4f948d217fdce2642b8f P 21a48e797c1942b4bfa349ac15116147 reported cstate change: term changed from 0 to 1, leader changed from <none> to 21a48e797c1942b4bfa349ac15116147 (127.24.227.194). New cstate: current_term: 1 leader_uuid: "21a48e797c1942b4bfa349ac15116147" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21a48e797c1942b4bfa349ac15116147" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 45731 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "78106638c5c545219d95183f82117c82" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 38423 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4a935543f574f50844743bce28cbab2" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 38023 } health_report { overall_health: UNKNOWN } } }
W20260923 23:14:27.744067 27560 tablet.cc:2369] T 139ebcfbfa8e4f948d217fdce2642b8f P 21a48e797c1942b4bfa349ac15116147: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 23 23:14:27 dist-test-slave-754n krb5kdc[27171](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205267, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.254@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
I20260923 23:14:27.766798 27739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:27.760409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:42395 (local address 127.0.0.1:50666)
0923 23:14:27.760562 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:27.760575 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:27.760653 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:27.760846 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:27.760847 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:27.761139 (+ 292us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:14:27.761175 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:27.761178 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:27.761800 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:27.761801 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:27.762310 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:27.762311 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:27.762646 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:27.762647 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:27.762705 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:27.763206 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:14:27.763211 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:14:27.764479 (+ 1268us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:14:27.765804 (+ 1325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:27.765811 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:27.765813 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:27.765989 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:27.766194 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:27.766195 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:27.766195 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:27.766248 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:27.766461 (+ 213us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:14:27.766467 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:14:27.766506 (+ 39us) client_negotiation.cc:769] Sending connection context
0923 23:14:27.766585 (+ 79us) client_negotiation.cc:240] Negotiation successful
0923 23:14:27.766692 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"mutex_wait_us":2,"thread_start_us":75,"threads_started":1}
I20260923 23:14:27.766803 27266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:27.760467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50666 (local address 127.24.227.254:42395)
0923 23:14:27.760531 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:27.760533 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:27.760639 (+ 106us) server_negotiation.cc:407] Connection header received
0923 23:14:27.760727 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:27.760728 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:27.760737 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:27.760841 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:27.761259 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:27.761722 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:27.762390 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:27.762561 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:27.762645 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:27.764566 (+ 1921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:27.764567 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:27.764567 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:27.764574 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:27.765713 (+ 1139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:27.766069 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:27.766070 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:27.766070 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:27.766123 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:27.766305 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:27.766306 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:27.766306 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:27.766403 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:27.766470 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:27.766609 (+ 139us) server_negotiation.cc:299] Negotiation successful
0923 23:14:27.766735 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":68,"server-negotiator.queue_time_us":41}
I20260923 23:14:27.768617 27215 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.24.227.254@KRBTEST.COM'} at 127.0.0.1:50666:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260923 23:14:27.768796 27215 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 23:14:27.770143 27353 tablet_service.cc:1467] Processing CreateTablet for tablet 8390301c815f4a2a949ce47aeb3f738a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=434d70c816fe44efb1588b980363ef47]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260923 23:14:27.770139 27493 tablet_service.cc:1467] Processing CreateTablet for tablet 8390301c815f4a2a949ce47aeb3f738a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=434d70c816fe44efb1588b980363ef47]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260923 23:14:27.770175 27633 tablet_service.cc:1467] Processing CreateTablet for tablet 8390301c815f4a2a949ce47aeb3f738a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=434d70c816fe44efb1588b980363ef47]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260923 23:14:27.770300 27633 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8390301c815f4a2a949ce47aeb3f738a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:14:27.770300 27493 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8390301c815f4a2a949ce47aeb3f738a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:14:27.770304 27353 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8390301c815f4a2a949ce47aeb3f738a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:14:27.771317 27722 tablet_bootstrap.cc:492] T 8390301c815f4a2a949ce47aeb3f738a P b4a935543f574f50844743bce28cbab2: Bootstrap starting.
I20260923 23:14:27.771638 27724 tablet_bootstrap.cc:492] T 8390301c815f4a2a949ce47aeb3f738a P 78106638c5c545219d95183f82117c82: Bootstrap starting.
I20260923 23:14:27.772027 27722 tablet_bootstrap.cc:654] T 8390301c815f4a2a949ce47aeb3f738a P b4a935543f574f50844743bce28cbab2: Neither blocks nor log segments found. Creating new log.
I20260923 23:14:27.772162 27724 tablet_bootstrap.cc:654] T 8390301c815f4a2a949ce47aeb3f738a P 78106638c5c545219d95183f82117c82: Neither blocks nor log segments found. Creating new log.
I20260923 23:14:27.772473 27723 tablet_bootstrap.cc:492] T 8390301c815f4a2a949ce47aeb3f738a P 21a48e797c1942b4bfa349ac15116147: Bootstrap starting.
I20260923 23:14:27.772697 27722 tablet_bootstrap.cc:492] T 8390301c815f4a2a949ce47aeb3f738a P b4a935543f574f50844743bce28cbab2: No bootstrap required, opened a new log
I20260923 23:14:27.772719 27724 tablet_bootstrap.cc:492] T 8390301c815f4a2a949ce47aeb3f738a P 78106638c5c545219d95183f82117c82: No bootstrap required, opened a new log
I20260923 23:14:27.772742 27722 ts_tablet_manager.cc:1397] T 8390301c815f4a2a949ce47aeb3f738a P b4a935543f574f50844743bce28cbab2: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260923 23:14:27.772753 27724 ts_tablet_manager.cc:1397] T 8390301c815f4a2a949ce47aeb3f738a P 78106638c5c545219d95183f82117c82: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260923 23:14:27.772817 27723 tablet_bootstrap.cc:654] T 8390301c815f4a2a949ce47aeb3f738a P 21a48e797c1942b4bfa349ac15116147: Neither blocks nor log segments found. Creating new log.
I20260923 23:14:27.772892 27724 raft_consensus.cc:348] T 8390301c815f4a2a949ce47aeb3f738a P 78106638c5c545219d95183f82117c82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4a935543f574f50844743bce28cbab2" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 38023 } } peers { permanent_uuid: "21a48e797c1942b4bfa349ac15116147" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 45731 } } peers { permanent_uuid: "78106638c5c545219d95183f82117c82" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 38423 } }
I20260923 23:14:27.772892 27722 raft_consensus.cc:348] T 8390301c815f4a2a949ce47aeb3f738a P b4a935543f574f50844743bce28cbab2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4a935543f574f50844743bce28cbab2" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 38023 } } peers { permanent_uuid: "21a48e797c1942b4bfa349ac15116147" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 45731 } } peers { permanent_uuid: "78106638c5c545219d95183f82117c82" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 38423 } }
I20260923 23:14:27.772951 27724 raft_consensus.cc:374] T 8390301c815f4a2a949ce47aeb3f738a P 78106638c5c545219d95183f82117c82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:14:27.772957 27722 raft_consensus.cc:374] T 8390301c815f4a2a949ce47aeb3f738a P b4a935543f574f50844743bce28cbab2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:14:27.772974 27724 raft_consensus.cc:729] T 8390301c815f4a2a949ce47aeb3f738a P 78106638c5c545219d95183f82117c82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78106638c5c545219d95183f82117c82, State: Initialized, Role: FOLLOWER
I20260923 23:14:27.772979 27722 raft_consensus.cc:729] T 8390301c815f4a2a949ce47aeb3f738a P b4a935543f574f50844743bce28cbab2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4a935543f574f50844743bce28cbab2, State: Initialized, Role: FOLLOWER
I20260923 23:14:27.773023 27724 consensus_queue.cc:260] T 8390301c815f4a2a949ce47aeb3f738a P 78106638c5c545219d95183f82117c82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4a935543f574f50844743bce28cbab2" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 38023 } } peers { permanent_uuid: "21a48e797c1942b4bfa349ac15116147" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 45731 } } peers { permanent_uuid: "78106638c5c545219d95183f82117c82" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 38423 } }
I20260923 23:14:27.773026 27722 consensus_queue.cc:260] T 8390301c815f4a2a949ce47aeb3f738a P b4a935543f574f50844743bce28cbab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4a935543f574f50844743bce28cbab2" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 38023 } } peers { permanent_uuid: "21a48e797c1942b4bfa349ac15116147" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 45731 } } peers { permanent_uuid: "78106638c5c545219d95183f82117c82" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 38423 } }
I20260923 23:14:27.773114 27724 ts_tablet_manager.cc:1428] T 8390301c815f4a2a949ce47aeb3f738a P 78106638c5c545219d95183f82117c82: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:27.773113 27722 ts_tablet_manager.cc:1428] T 8390301c815f4a2a949ce47aeb3f738a P b4a935543f574f50844743bce28cbab2: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:27.773176 27730 tablet_replica.cc:439] T 8390301c815f4a2a949ce47aeb3f738a P 78106638c5c545219d95183f82117c82: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4a935543f574f50844743bce28cbab2" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 38023 } } peers { permanent_uuid: "21a48e797c1942b4bfa349ac15116147" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 45731 } } peers { permanent_uuid: "78106638c5c545219d95183f82117c82" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 38423 } } }
I20260923 23:14:27.773273 27730 tablet_replica.cc:442] T 8390301c815f4a2a949ce47aeb3f738a P 78106638c5c545219d95183f82117c82: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 23:14:27.773412 27728 tablet_replica.cc:439] T 8390301c815f4a2a949ce47aeb3f738a P b4a935543f574f50844743bce28cbab2: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4a935543f574f50844743bce28cbab2" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 38023 } } peers { permanent_uuid: "21a48e797c1942b4bfa349ac15116147" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 45731 } } peers { permanent_uuid: "78106638c5c545219d95183f82117c82" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 38423 } } }
I20260923 23:14:27.773495 27728 tablet_replica.cc:442] T 8390301c815f4a2a949ce47aeb3f738a P b4a935543f574f50844743bce28cbab2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 23:14:27.773784 27723 tablet_bootstrap.cc:492] T 8390301c815f4a2a949ce47aeb3f738a P 21a48e797c1942b4bfa349ac15116147: No bootstrap required, opened a new log
I20260923 23:14:27.773837 27723 ts_tablet_manager.cc:1397] T 8390301c815f4a2a949ce47aeb3f738a P 21a48e797c1942b4bfa349ac15116147: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260923 23:14:27.773978 27723 raft_consensus.cc:348] T 8390301c815f4a2a949ce47aeb3f738a P 21a48e797c1942b4bfa349ac15116147 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4a935543f574f50844743bce28cbab2" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 38023 } } peers { permanent_uuid: "21a48e797c1942b4bfa349ac15116147" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 45731 } } peers { permanent_uuid: "78106638c5c545219d95183f82117c82" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 38423 } }
I20260923 23:14:27.774065 27723 raft_consensus.cc:374] T 8390301c815f4a2a949ce47aeb3f738a P 21a48e797c1942b4bfa349ac15116147 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:14:27.774087 27723 raft_consensus.cc:729] T 8390301c815f4a2a949ce47aeb3f738a P 21a48e797c1942b4bfa349ac15116147 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21a48e797c1942b4bfa349ac15116147, State: Initialized, Role: FOLLOWER
I20260923 23:14:27.774139 27723 consensus_queue.cc:260] T 8390301c815f4a2a949ce47aeb3f738a P 21a48e797c1942b4bfa349ac15116147 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4a935543f574f50844743bce28cbab2" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 38023 } } peers { permanent_uuid: "21a48e797c1942b4bfa349ac15116147" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 45731 } } peers { permanent_uuid: "78106638c5c545219d95183f82117c82" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 38423 } }
I20260923 23:14:27.774220 27723 ts_tablet_manager.cc:1428] T 8390301c815f4a2a949ce47aeb3f738a P 21a48e797c1942b4bfa349ac15116147: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:27.774258 27729 tablet_replica.cc:439] T 8390301c815f4a2a949ce47aeb3f738a P 21a48e797c1942b4bfa349ac15116147: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4a935543f574f50844743bce28cbab2" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 38023 } } peers { permanent_uuid: "21a48e797c1942b4bfa349ac15116147" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 45731 } } peers { permanent_uuid: "78106638c5c545219d95183f82117c82" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 38423 } } }
I20260923 23:14:27.774333 27729 tablet_replica.cc:442] T 8390301c815f4a2a949ce47aeb3f738a P 21a48e797c1942b4bfa349ac15116147: This TxnStatusTablet replica's current role is: FOLLOWER
W20260923 23:14:27.840456 27420 tablet.cc:2369] T 139ebcfbfa8e4f948d217fdce2642b8f P b4a935543f574f50844743bce28cbab2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 23:14:27.894569 27700 tablet.cc:2369] T 139ebcfbfa8e4f948d217fdce2642b8f P 78106638c5c545219d95183f82117c82: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 23:14:27.931488 27729 raft_consensus.cc:482] T 8390301c815f4a2a949ce47aeb3f738a P 21a48e797c1942b4bfa349ac15116147 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 23:14:27.931576 27729 raft_consensus.cc:504] T 8390301c815f4a2a949ce47aeb3f738a P 21a48e797c1942b4bfa349ac15116147 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4a935543f574f50844743bce28cbab2" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 38023 } } peers { permanent_uuid: "21a48e797c1942b4bfa349ac15116147" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 45731 } } peers { permanent_uuid: "78106638c5c545219d95183f82117c82" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 38423 } }
I20260923 23:14:27.931727 27729 leader_election.cc:288] T 8390301c815f4a2a949ce47aeb3f738a P 21a48e797c1942b4bfa349ac15116147 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b4a935543f574f50844743bce28cbab2 (127.24.227.193:38023), 78106638c5c545219d95183f82117c82 (127.24.227.195:38423)
I20260923 23:14:27.931985 27653 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8390301c815f4a2a949ce47aeb3f738a" candidate_uuid: "21a48e797c1942b4bfa349ac15116147" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78106638c5c545219d95183f82117c82" is_pre_election: true
I20260923 23:14:27.931989 27373 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8390301c815f4a2a949ce47aeb3f738a" candidate_uuid: "21a48e797c1942b4bfa349ac15116147" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4a935543f574f50844743bce28cbab2" is_pre_election: true
I20260923 23:14:27.932070 27653 raft_consensus.cc:2454] T 8390301c815f4a2a949ce47aeb3f738a P 78106638c5c545219d95183f82117c82 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 21a48e797c1942b4bfa349ac15116147 in term 0.
I20260923 23:14:27.932104 27373 raft_consensus.cc:2454] T 8390301c815f4a2a949ce47aeb3f738a P b4a935543f574f50844743bce28cbab2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 21a48e797c1942b4bfa349ac15116147 in term 0.
I20260923 23:14:27.932233 27446 leader_election.cc:302] T 8390301c815f4a2a949ce47aeb3f738a P 21a48e797c1942b4bfa349ac15116147 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21a48e797c1942b4bfa349ac15116147, 78106638c5c545219d95183f82117c82; no voters:
I20260923 23:14:27.932344 27729 raft_consensus.cc:2781] T 8390301c815f4a2a949ce47aeb3f738a P 21a48e797c1942b4bfa349ac15116147 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 23:14:27.932390 27729 raft_consensus.cc:482] T 8390301c815f4a2a949ce47aeb3f738a P 21a48e797c1942b4bfa349ac15116147 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 23:14:27.932416 27729 raft_consensus.cc:3037] T 8390301c815f4a2a949ce47aeb3f738a P 21a48e797c1942b4bfa349ac15116147 [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:14:27.933079 27729 raft_consensus.cc:504] T 8390301c815f4a2a949ce47aeb3f738a P 21a48e797c1942b4bfa349ac15116147 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4a935543f574f50844743bce28cbab2" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 38023 } } peers { permanent_uuid: "21a48e797c1942b4bfa349ac15116147" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 45731 } } peers { permanent_uuid: "78106638c5c545219d95183f82117c82" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 38423 } }
I20260923 23:14:27.933216 27729 leader_election.cc:288] T 8390301c815f4a2a949ce47aeb3f738a P 21a48e797c1942b4bfa349ac15116147 [CANDIDATE]: Term 1 election: Requested vote from peers b4a935543f574f50844743bce28cbab2 (127.24.227.193:38023), 78106638c5c545219d95183f82117c82 (127.24.227.195:38423)
I20260923 23:14:27.933341 27653 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8390301c815f4a2a949ce47aeb3f738a" candidate_uuid: "21a48e797c1942b4bfa349ac15116147" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78106638c5c545219d95183f82117c82"
I20260923 23:14:27.933349 27373 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8390301c815f4a2a949ce47aeb3f738a" candidate_uuid: "21a48e797c1942b4bfa349ac15116147" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4a935543f574f50844743bce28cbab2"
I20260923 23:14:27.933411 27373 raft_consensus.cc:3037] T 8390301c815f4a2a949ce47aeb3f738a P b4a935543f574f50844743bce28cbab2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:14:27.933422 27653 raft_consensus.cc:3037] T 8390301c815f4a2a949ce47aeb3f738a P 78106638c5c545219d95183f82117c82 [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:14:27.934013 27653 raft_consensus.cc:2454] T 8390301c815f4a2a949ce47aeb3f738a P 78106638c5c545219d95183f82117c82 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21a48e797c1942b4bfa349ac15116147 in term 1.
I20260923 23:14:27.934021 27373 raft_consensus.cc:2454] T 8390301c815f4a2a949ce47aeb3f738a P b4a935543f574f50844743bce28cbab2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21a48e797c1942b4bfa349ac15116147 in term 1.
I20260923 23:14:27.934190 27444 leader_election.cc:302] T 8390301c815f4a2a949ce47aeb3f738a P 21a48e797c1942b4bfa349ac15116147 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21a48e797c1942b4bfa349ac15116147, b4a935543f574f50844743bce28cbab2; no voters:
I20260923 23:14:27.934295 27729 raft_consensus.cc:2781] T 8390301c815f4a2a949ce47aeb3f738a P 21a48e797c1942b4bfa349ac15116147 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 23:14:27.934379 27729 raft_consensus.cc:686] T 8390301c815f4a2a949ce47aeb3f738a P 21a48e797c1942b4bfa349ac15116147 [term 1 LEADER]: Becoming Leader. State: Replica: 21a48e797c1942b4bfa349ac15116147, State: Running, Role: LEADER
I20260923 23:14:27.934437 27729 consensus_queue.cc:237] T 8390301c815f4a2a949ce47aeb3f738a P 21a48e797c1942b4bfa349ac15116147 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4a935543f574f50844743bce28cbab2" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 38023 } } peers { permanent_uuid: "21a48e797c1942b4bfa349ac15116147" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 45731 } } peers { permanent_uuid: "78106638c5c545219d95183f82117c82" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 38423 } }
I20260923 23:14:27.934770 27732 tablet_replica.cc:439] T 8390301c815f4a2a949ce47aeb3f738a P 21a48e797c1942b4bfa349ac15116147: TxnStatusTablet state changed. Reason: New leader 21a48e797c1942b4bfa349ac15116147. Latest consensus state: current_term: 1 leader_uuid: "21a48e797c1942b4bfa349ac15116147" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4a935543f574f50844743bce28cbab2" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 38023 } } peers { permanent_uuid: "21a48e797c1942b4bfa349ac15116147" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 45731 } } peers { permanent_uuid: "78106638c5c545219d95183f82117c82" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 38423 } } }
I20260923 23:14:27.934861 27732 tablet_replica.cc:442] T 8390301c815f4a2a949ce47aeb3f738a P 21a48e797c1942b4bfa349ac15116147: This TxnStatusTablet replica's current role is: LEADER
I20260923 23:14:27.935088 27744 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260923 23:14:27.935212 27744 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260923 23:14:27.935060 27216 catalog_manager.cc:5495] T 8390301c815f4a2a949ce47aeb3f738a P 21a48e797c1942b4bfa349ac15116147 reported cstate change: term changed from 0 to 1, leader changed from <none> to 21a48e797c1942b4bfa349ac15116147 (127.24.227.194). New cstate: current_term: 1 leader_uuid: "21a48e797c1942b4bfa349ac15116147" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4a935543f574f50844743bce28cbab2" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 38023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "21a48e797c1942b4bfa349ac15116147" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 45731 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "78106638c5c545219d95183f82117c82" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 38423 } health_report { overall_health: UNKNOWN } } }
I20260923 23:14:27.954298 27739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:27.951337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.194:45731 (local address 127.0.0.1:43924)
0923 23:14:27.951414 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:27.951426 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:27.951479 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:27.951761 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:27.951762 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:27.951763 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0923 23:14:27.951807 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:27.951809 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:27.952721 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:27.952722 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:27.953367 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:27.953370 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:27.953832 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:27.953833 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:27.953878 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:27.953891 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0923 23:14:27.954164 (+ 273us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0923 23:14:27.954170 (+ 6us) client_negotiation.cc:769] Sending connection context
0923 23:14:27.954198 (+ 28us) client_negotiation.cc:240] Negotiation successful
0923 23:14:27.954241 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260923 23:14:27.954298 27721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:27.951548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43924 (local address 127.24.227.194:45731)
0923 23:14:27.951613 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:27.951615 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:27.951624 (+ 9us) server_negotiation.cc:407] Connection header received
0923 23:14:27.951655 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:27.951656 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:27.951677 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:27.951750 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 23:14:27.951860 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:27.952643 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:27.953443 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:27.953756 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:27.953828 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:27.953975 (+ 147us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 23:14:27.954100 (+ 125us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 23:14:27.954178 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:27.954201 (+ 23us) server_negotiation.cc:299] Negotiation successful
0923 23:14:27.954225 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260923 23:14:27.955978 27653 raft_consensus.cc:1261] T 8390301c815f4a2a949ce47aeb3f738a P 78106638c5c545219d95183f82117c82 [term 1 FOLLOWER]: Refusing update from remote peer 21a48e797c1942b4bfa349ac15116147: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 23:14:27.956017 27373 raft_consensus.cc:1261] T 8390301c815f4a2a949ce47aeb3f738a P b4a935543f574f50844743bce28cbab2 [term 1 FOLLOWER]: Refusing update from remote peer 21a48e797c1942b4bfa349ac15116147: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 23:14:27.956302 27732 consensus_queue.cc:1035] T 8390301c815f4a2a949ce47aeb3f738a P 21a48e797c1942b4bfa349ac15116147 [LEADER]: Connected to new peer: Peer: permanent_uuid: "78106638c5c545219d95183f82117c82" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 38423 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 23:14:27.956439 27729 consensus_queue.cc:1035] T 8390301c815f4a2a949ce47aeb3f738a P 21a48e797c1942b4bfa349ac15116147 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4a935543f574f50844743bce28cbab2" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 38023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 23:14:27.957646 27730 tablet_replica.cc:439] T 8390301c815f4a2a949ce47aeb3f738a P 78106638c5c545219d95183f82117c82: TxnStatusTablet state changed. Reason: New leader 21a48e797c1942b4bfa349ac15116147. Latest consensus state: current_term: 1 leader_uuid: "21a48e797c1942b4bfa349ac15116147" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4a935543f574f50844743bce28cbab2" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 38023 } } peers { permanent_uuid: "21a48e797c1942b4bfa349ac15116147" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 45731 } } peers { permanent_uuid: "78106638c5c545219d95183f82117c82" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 38423 } } }
I20260923 23:14:27.957670 27728 tablet_replica.cc:439] T 8390301c815f4a2a949ce47aeb3f738a P b4a935543f574f50844743bce28cbab2: TxnStatusTablet state changed. Reason: New leader 21a48e797c1942b4bfa349ac15116147. Latest consensus state: current_term: 1 leader_uuid: "21a48e797c1942b4bfa349ac15116147" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4a935543f574f50844743bce28cbab2" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 38023 } } peers { permanent_uuid: "21a48e797c1942b4bfa349ac15116147" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 45731 } } peers { permanent_uuid: "78106638c5c545219d95183f82117c82" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 38423 } } }
I20260923 23:14:27.957744 27730 tablet_replica.cc:442] T 8390301c815f4a2a949ce47aeb3f738a P 78106638c5c545219d95183f82117c82: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 23:14:27.957765 27728 tablet_replica.cc:442] T 8390301c815f4a2a949ce47aeb3f738a P b4a935543f574f50844743bce28cbab2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 23:14:27.958650 27732 tablet_replica.cc:439] T 8390301c815f4a2a949ce47aeb3f738a P 21a48e797c1942b4bfa349ac15116147: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "21a48e797c1942b4bfa349ac15116147" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4a935543f574f50844743bce28cbab2" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 38023 } } peers { permanent_uuid: "21a48e797c1942b4bfa349ac15116147" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 45731 } } peers { permanent_uuid: "78106638c5c545219d95183f82117c82" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 38423 } } }
I20260923 23:14:27.958747 27732 tablet_replica.cc:442] T 8390301c815f4a2a949ce47aeb3f738a P 21a48e797c1942b4bfa349ac15116147: This TxnStatusTablet replica's current role is: LEADER
I20260923 23:14:27.958781 27730 tablet_replica.cc:439] T 8390301c815f4a2a949ce47aeb3f738a P 78106638c5c545219d95183f82117c82: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "21a48e797c1942b4bfa349ac15116147" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4a935543f574f50844743bce28cbab2" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 38023 } } peers { permanent_uuid: "21a48e797c1942b4bfa349ac15116147" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 45731 } } peers { permanent_uuid: "78106638c5c545219d95183f82117c82" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 38423 } } }
I20260923 23:14:27.958801 27732 tablet_replica.cc:439] T 8390301c815f4a2a949ce47aeb3f738a P 21a48e797c1942b4bfa349ac15116147: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "21a48e797c1942b4bfa349ac15116147" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4a935543f574f50844743bce28cbab2" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 38023 } } peers { permanent_uuid: "21a48e797c1942b4bfa349ac15116147" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 45731 } } peers { permanent_uuid: "78106638c5c545219d95183f82117c82" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 38423 } } }
I20260923 23:14:27.958853 27730 tablet_replica.cc:442] T 8390301c815f4a2a949ce47aeb3f738a P 78106638c5c545219d95183f82117c82: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 23:14:27.958853 27732 tablet_replica.cc:442] T 8390301c815f4a2a949ce47aeb3f738a P 21a48e797c1942b4bfa349ac15116147: This TxnStatusTablet replica's current role is: LEADER
I20260923 23:14:27.958997 27728 tablet_replica.cc:439] T 8390301c815f4a2a949ce47aeb3f738a P b4a935543f574f50844743bce28cbab2: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "21a48e797c1942b4bfa349ac15116147" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4a935543f574f50844743bce28cbab2" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 38023 } } peers { permanent_uuid: "21a48e797c1942b4bfa349ac15116147" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 45731 } } peers { permanent_uuid: "78106638c5c545219d95183f82117c82" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 38423 } } }
I20260923 23:14:27.959089 27728 tablet_replica.cc:442] T 8390301c815f4a2a949ce47aeb3f738a P b4a935543f574f50844743bce28cbab2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 23:14:27.959205 27747 mvcc.cc:205] Tried to move back new op lower bound from 7332680777544728576 to 7332680777459412992. Current Snapshot: MvccSnapshot[applied={T|T < 7332680777544728576}]
I20260923 23:14:27.959339 27748 mvcc.cc:205] Tried to move back new op lower bound from 7332680777544728576 to 7332680777459412992. Current Snapshot: MvccSnapshot[applied={T|T < 7332680777544728576}]
I20260923 23:14:27.960022 27749 mvcc.cc:205] Tried to move back new op lower bound from 7332680777544728576 to 7332680777459412992. Current Snapshot: MvccSnapshot[applied={T|T < 7332680777544728576}]
I20260923 23:14:27.963392 27721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:27.961343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43928 (local address 127.24.227.194:45731)
0923 23:14:27.961394 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:27.961396 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:27.961404 (+ 8us) server_negotiation.cc:407] Connection header received
0923 23:14:27.961426 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:27.961426 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:27.961436 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:27.961483 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 23:14:27.961579 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:27.962008 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:27.962755 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:27.962961 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:27.963034 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:27.963185 (+ 151us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 23:14:27.963214 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 23:14:27.963260 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:27.963311 (+ 51us) server_negotiation.cc:299] Negotiation successful
0923 23:14:27.963329 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260923 23:14:27.967535 27721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:27.965531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.194:57917 (local address 127.24.227.194:45731)
0923 23:14:27.965573 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:27.965575 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:27.965609 (+ 34us) server_negotiation.cc:407] Connection header received
0923 23:14:27.965678 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:27.965679 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:27.965686 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:27.965742 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 23:14:27.965839 (+ 97us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:27.966234 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:27.966884 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:27.967061 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:27.967125 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:27.967329 (+ 204us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 23:14:27.967353 (+ 24us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 23:14:27.967394 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:27.967442 (+ 48us) server_negotiation.cc:299] Negotiation successful
0923 23:14:27.967458 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260923 23:14:27.967535 27565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:27.965486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.194:45731 (local address 127.24.227.194:57917)
0923 23:14:27.965573 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:27.965582 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:27.965622 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:27.965745 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:27.965745 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:27.965746 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0923 23:14:27.965788 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:27.965790 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:27.966305 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:27.966306 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:27.966796 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:27.966797 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:27.967123 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:27.967124 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:27.967183 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:27.967193 (+ 10us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0923 23:14:27.967393 (+ 200us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0923 23:14:27.967394 (+ 1us) client_negotiation.cc:769] Sending connection context
0923 23:14:27.967439 (+ 45us) client_negotiation.cc:240] Negotiation successful
0923 23:14:27.967469 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260923 23:14:27.969297 27373 raft_consensus.cc:1261] T 139ebcfbfa8e4f948d217fdce2642b8f P b4a935543f574f50844743bce28cbab2 [term 1 FOLLOWER]: Refusing update from remote peer 21a48e797c1942b4bfa349ac15116147: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 23:14:27.969544 27758 consensus_queue.cc:1035] T 139ebcfbfa8e4f948d217fdce2642b8f P 21a48e797c1942b4bfa349ac15116147 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4a935543f574f50844743bce28cbab2" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 38023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 23:14:27.970391 27653 raft_consensus.cc:1261] T 139ebcfbfa8e4f948d217fdce2642b8f P 78106638c5c545219d95183f82117c82 [term 1 FOLLOWER]: Refusing update from remote peer 21a48e797c1942b4bfa349ac15116147: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 23:14:27.970553 27758 consensus_queue.cc:1035] T 139ebcfbfa8e4f948d217fdce2642b8f P 21a48e797c1942b4bfa349ac15116147 [LEADER]: Connected to new peer: Peer: permanent_uuid: "78106638c5c545219d95183f82117c82" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 38423 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 23:14:27.984138 27216 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50660:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 23:14:27.984246 27216 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50660:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 23:14:27.984975 27216 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e6ed2aa900084446a84ca867a45396ae: Sending DeleteTablet for 3 replicas of tablet 139ebcfbfa8e4f948d217fdce2642b8f
I20260923 23:14:27.985252 27493 tablet_service.cc:1514] Processing DeleteTablet for tablet 139ebcfbfa8e4f948d217fdce2642b8f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 23:14:27 UTC) from {username='kudu', principal='kudu/127.24.227.254@KRBTEST.COM'} at 127.0.0.1:43922
I20260923 23:14:27.985255 27633 tablet_service.cc:1514] Processing DeleteTablet for tablet 139ebcfbfa8e4f948d217fdce2642b8f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 23:14:27 UTC) from {username='kudu', principal='kudu/127.24.227.254@KRBTEST.COM'} at 127.0.0.1:51296
I20260923 23:14:27.985343 27353 tablet_service.cc:1514] Processing DeleteTablet for tablet 139ebcfbfa8e4f948d217fdce2642b8f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 23:14:27 UTC) from {username='kudu', principal='kudu/127.24.227.254@KRBTEST.COM'} at 127.0.0.1:41908
I20260923 23:14:27.985522 27766 tablet_replica.cc:330] T 139ebcfbfa8e4f948d217fdce2642b8f P b4a935543f574f50844743bce28cbab2: stopping tablet replica
I20260923 23:14:27.985615 27766 raft_consensus.cc:2229] T 139ebcfbfa8e4f948d217fdce2642b8f P b4a935543f574f50844743bce28cbab2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 23:14:27.985674 27766 raft_consensus.cc:2258] T 139ebcfbfa8e4f948d217fdce2642b8f P b4a935543f574f50844743bce28cbab2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 23:14:27.985702 27216 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50660:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 23:14:27.985867 27216 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 23:14:27.986231 27766 ts_tablet_manager.cc:1905] T 139ebcfbfa8e4f948d217fdce2642b8f P b4a935543f574f50844743bce28cbab2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 23:14:27.986212 27764 tablet_replica.cc:330] T 139ebcfbfa8e4f948d217fdce2642b8f P 21a48e797c1942b4bfa349ac15116147: stopping tablet replica
I20260923 23:14:27.987051 27353 tablet_service.cc:1467] Processing CreateTablet for tablet 9c1449274cde41d5b675dd3c7a39090d (DEFAULT_TABLE table=test-table [id=8146c81c780c42569bc0f24e9194a9fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 23:14:27.987053 27633 tablet_service.cc:1467] Processing CreateTablet for tablet 9c1449274cde41d5b675dd3c7a39090d (DEFAULT_TABLE table=test-table [id=8146c81c780c42569bc0f24e9194a9fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 23:14:27.987159 27633 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c1449274cde41d5b675dd3c7a39090d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:14:27.987274 27493 tablet_service.cc:1467] Processing CreateTablet for tablet 9c1449274cde41d5b675dd3c7a39090d (DEFAULT_TABLE table=test-table [id=8146c81c780c42569bc0f24e9194a9fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 23:14:27.987399 27493 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c1449274cde41d5b675dd3c7a39090d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:14:27.987978 27765 tablet_replica.cc:330] T 139ebcfbfa8e4f948d217fdce2642b8f P 78106638c5c545219d95183f82117c82: stopping tablet replica
I20260923 23:14:27.988123 27765 raft_consensus.cc:2229] T 139ebcfbfa8e4f948d217fdce2642b8f P 78106638c5c545219d95183f82117c82 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 23:14:27.988214 27765 raft_consensus.cc:2258] T 139ebcfbfa8e4f948d217fdce2642b8f P 78106638c5c545219d95183f82117c82 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 23:14:27.988600 27724 tablet_bootstrap.cc:492] T 9c1449274cde41d5b675dd3c7a39090d P 78106638c5c545219d95183f82117c82: Bootstrap starting.
I20260923 23:14:27.988657 27723 tablet_bootstrap.cc:492] T 9c1449274cde41d5b675dd3c7a39090d P 21a48e797c1942b4bfa349ac15116147: Bootstrap starting.
I20260923 23:14:27.989018 27724 tablet_bootstrap.cc:654] T 9c1449274cde41d5b675dd3c7a39090d P 78106638c5c545219d95183f82117c82: Neither blocks nor log segments found. Creating new log.
I20260923 23:14:27.989029 27723 tablet_bootstrap.cc:654] T 9c1449274cde41d5b675dd3c7a39090d P 21a48e797c1942b4bfa349ac15116147: Neither blocks nor log segments found. Creating new log.
I20260923 23:14:27.989516 27353 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c1449274cde41d5b675dd3c7a39090d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:14:27.989603 27765 ts_tablet_manager.cc:1905] T 139ebcfbfa8e4f948d217fdce2642b8f P 78106638c5c545219d95183f82117c82: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 23:14:27.989918 27724 tablet_bootstrap.cc:492] T 9c1449274cde41d5b675dd3c7a39090d P 78106638c5c545219d95183f82117c82: No bootstrap required, opened a new log
I20260923 23:14:27.989961 27724 ts_tablet_manager.cc:1397] T 9c1449274cde41d5b675dd3c7a39090d P 78106638c5c545219d95183f82117c82: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260923 23:14:27.990119 27724 raft_consensus.cc:348] T 9c1449274cde41d5b675dd3c7a39090d P 78106638c5c545219d95183f82117c82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4a935543f574f50844743bce28cbab2" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 38023 } } peers { permanent_uuid: "21a48e797c1942b4bfa349ac15116147" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 45731 } } peers { permanent_uuid: "78106638c5c545219d95183f82117c82" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 38423 } }
I20260923 23:14:27.990171 27724 raft_consensus.cc:374] T 9c1449274cde41d5b675dd3c7a39090d P 78106638c5c545219d95183f82117c82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:14:27.990188 27724 raft_consensus.cc:729] T 9c1449274cde41d5b675dd3c7a39090d P 78106638c5c545219d95183f82117c82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78106638c5c545219d95183f82117c82, State: Initialized, Role: FOLLOWER
I20260923 23:14:27.990231 27724 consensus_queue.cc:260] T 9c1449274cde41d5b675dd3c7a39090d P 78106638c5c545219d95183f82117c82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4a935543f574f50844743bce28cbab2" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 38023 } } peers { permanent_uuid: "21a48e797c1942b4bfa349ac15116147" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 45731 } } peers { permanent_uuid: "78106638c5c545219d95183f82117c82" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 38423 } }
I20260923 23:14:27.990305 27724 ts_tablet_manager.cc:1428] T 9c1449274cde41d5b675dd3c7a39090d P 78106638c5c545219d95183f82117c82: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:27.990829 27722 tablet_bootstrap.cc:492] T 9c1449274cde41d5b675dd3c7a39090d P b4a935543f574f50844743bce28cbab2: Bootstrap starting.
I20260923 23:14:27.991078 27765 ts_tablet_manager.cc:1918] T 139ebcfbfa8e4f948d217fdce2642b8f P 78106638c5c545219d95183f82117c82: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 23:14:27.991146 27765 log.cc:1196] T 139ebcfbfa8e4f948d217fdce2642b8f P 78106638c5c545219d95183f82117c82: Deleting WAL directory at /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/minicluster-data/ts-2/wal/wals/139ebcfbfa8e4f948d217fdce2642b8f
I20260923 23:14:27.991211 27722 tablet_bootstrap.cc:654] T 9c1449274cde41d5b675dd3c7a39090d P b4a935543f574f50844743bce28cbab2: Neither blocks nor log segments found. Creating new log.
I20260923 23:14:27.991441 27765 ts_tablet_manager.cc:1939] T 139ebcfbfa8e4f948d217fdce2642b8f P 78106638c5c545219d95183f82117c82: Deleting consensus metadata
I20260923 23:14:27.991843 27202 catalog_manager.cc:4841] TS 78106638c5c545219d95183f82117c82 (127.24.227.195:38423): tablet 139ebcfbfa8e4f948d217fdce2642b8f (table test-table [id=3e83e0fac380470cbbbf6202ea910830]) successfully deleted
I20260923 23:14:27.991921 27723 tablet_bootstrap.cc:492] T 9c1449274cde41d5b675dd3c7a39090d P 21a48e797c1942b4bfa349ac15116147: No bootstrap required, opened a new log
I20260923 23:14:27.991959 27723 ts_tablet_manager.cc:1397] T 9c1449274cde41d5b675dd3c7a39090d P 21a48e797c1942b4bfa349ac15116147: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260923 23:14:27.992089 27723 raft_consensus.cc:348] T 9c1449274cde41d5b675dd3c7a39090d P 21a48e797c1942b4bfa349ac15116147 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4a935543f574f50844743bce28cbab2" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 38023 } } peers { permanent_uuid: "21a48e797c1942b4bfa349ac15116147" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 45731 } } peers { permanent_uuid: "78106638c5c545219d95183f82117c82" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 38423 } }
I20260923 23:14:27.992167 27723 raft_consensus.cc:374] T 9c1449274cde41d5b675dd3c7a39090d P 21a48e797c1942b4bfa349ac15116147 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:14:27.992190 27723 raft_consensus.cc:729] T 9c1449274cde41d5b675dd3c7a39090d P 21a48e797c1942b4bfa349ac15116147 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21a48e797c1942b4bfa349ac15116147, State: Initialized, Role: FOLLOWER
I20260923 23:14:27.992246 27723 consensus_queue.cc:260] T 9c1449274cde41d5b675dd3c7a39090d P 21a48e797c1942b4bfa349ac15116147 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4a935543f574f50844743bce28cbab2" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 38023 } } peers { permanent_uuid: "21a48e797c1942b4bfa349ac15116147" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 45731 } } peers { permanent_uuid: "78106638c5c545219d95183f82117c82" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 38423 } }
I20260923 23:14:27.992347 27723 ts_tablet_manager.cc:1428] T 9c1449274cde41d5b675dd3c7a39090d P 21a48e797c1942b4bfa349ac15116147: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:27.993119 27766 ts_tablet_manager.cc:1918] T 139ebcfbfa8e4f948d217fdce2642b8f P b4a935543f574f50844743bce28cbab2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 23:14:27.993204 27766 log.cc:1196] T 139ebcfbfa8e4f948d217fdce2642b8f P b4a935543f574f50844743bce28cbab2: Deleting WAL directory at /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/minicluster-data/ts-0/wal/wals/139ebcfbfa8e4f948d217fdce2642b8f
I20260923 23:14:27.993430 27766 ts_tablet_manager.cc:1939] T 139ebcfbfa8e4f948d217fdce2642b8f P b4a935543f574f50844743bce28cbab2: Deleting consensus metadata
I20260923 23:14:27.993592 27764 raft_consensus.cc:2229] T 139ebcfbfa8e4f948d217fdce2642b8f P 21a48e797c1942b4bfa349ac15116147 [term 1 LEADER]: Raft consensus shutting down.
I20260923 23:14:27.993705 27764 raft_consensus.cc:2258] T 139ebcfbfa8e4f948d217fdce2642b8f P 21a48e797c1942b4bfa349ac15116147 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 23:14:27.993882 27200 catalog_manager.cc:4841] TS b4a935543f574f50844743bce28cbab2 (127.24.227.193:38023): tablet 139ebcfbfa8e4f948d217fdce2642b8f (table test-table [id=3e83e0fac380470cbbbf6202ea910830]) successfully deleted
I20260923 23:14:27.994190 27722 tablet_bootstrap.cc:492] T 9c1449274cde41d5b675dd3c7a39090d P b4a935543f574f50844743bce28cbab2: No bootstrap required, opened a new log
I20260923 23:14:27.994240 27722 ts_tablet_manager.cc:1397] T 9c1449274cde41d5b675dd3c7a39090d P b4a935543f574f50844743bce28cbab2: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260923 23:14:27.994225 27764 ts_tablet_manager.cc:1905] T 139ebcfbfa8e4f948d217fdce2642b8f P 21a48e797c1942b4bfa349ac15116147: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 23:14:27.994379 27722 raft_consensus.cc:348] T 9c1449274cde41d5b675dd3c7a39090d P b4a935543f574f50844743bce28cbab2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4a935543f574f50844743bce28cbab2" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 38023 } } peers { permanent_uuid: "21a48e797c1942b4bfa349ac15116147" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 45731 } } peers { permanent_uuid: "78106638c5c545219d95183f82117c82" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 38423 } }
I20260923 23:14:27.994444 27722 raft_consensus.cc:374] T 9c1449274cde41d5b675dd3c7a39090d P b4a935543f574f50844743bce28cbab2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:14:27.994464 27722 raft_consensus.cc:729] T 9c1449274cde41d5b675dd3c7a39090d P b4a935543f574f50844743bce28cbab2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4a935543f574f50844743bce28cbab2, State: Initialized, Role: FOLLOWER
I20260923 23:14:27.994506 27722 consensus_queue.cc:260] T 9c1449274cde41d5b675dd3c7a39090d P b4a935543f574f50844743bce28cbab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4a935543f574f50844743bce28cbab2" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 38023 } } peers { permanent_uuid: "21a48e797c1942b4bfa349ac15116147" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 45731 } } peers { permanent_uuid: "78106638c5c545219d95183f82117c82" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 38423 } }
I20260923 23:14:27.994670 27722 ts_tablet_manager.cc:1428] T 9c1449274cde41d5b675dd3c7a39090d P b4a935543f574f50844743bce28cbab2: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:27.995850 27764 ts_tablet_manager.cc:1918] T 139ebcfbfa8e4f948d217fdce2642b8f P 21a48e797c1942b4bfa349ac15116147: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 23:14:27.995956 27764 log.cc:1196] T 139ebcfbfa8e4f948d217fdce2642b8f P 21a48e797c1942b4bfa349ac15116147: Deleting WAL directory at /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/minicluster-data/ts-1/wal/wals/139ebcfbfa8e4f948d217fdce2642b8f
I20260923 23:14:27.996182 27764 ts_tablet_manager.cc:1939] T 139ebcfbfa8e4f948d217fdce2642b8f P 21a48e797c1942b4bfa349ac15116147: Deleting consensus metadata
I20260923 23:14:27.996548 27203 catalog_manager.cc:4841] TS 21a48e797c1942b4bfa349ac15116147 (127.24.227.194:45731): tablet 139ebcfbfa8e4f948d217fdce2642b8f (table test-table [id=3e83e0fac380470cbbbf6202ea910830]) successfully deleted
I20260923 23:14:28.116006 27728 raft_consensus.cc:482] T 9c1449274cde41d5b675dd3c7a39090d P b4a935543f574f50844743bce28cbab2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 23:14:28.116117 27728 raft_consensus.cc:504] T 9c1449274cde41d5b675dd3c7a39090d P b4a935543f574f50844743bce28cbab2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4a935543f574f50844743bce28cbab2" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 38023 } } peers { permanent_uuid: "21a48e797c1942b4bfa349ac15116147" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 45731 } } peers { permanent_uuid: "78106638c5c545219d95183f82117c82" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 38423 } }
I20260923 23:14:28.116446 27728 leader_election.cc:288] T 9c1449274cde41d5b675dd3c7a39090d P b4a935543f574f50844743bce28cbab2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 21a48e797c1942b4bfa349ac15116147 (127.24.227.194:45731), 78106638c5c545219d95183f82117c82 (127.24.227.195:38423)
I20260923 23:14:28.119884 27720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:28.116721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.193:34725 (local address 127.24.227.195:38423)
0923 23:14:28.116773 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:28.116775 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:28.116864 (+ 89us) server_negotiation.cc:407] Connection header received
0923 23:14:28.116981 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:28.116982 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:28.116991 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:28.117018 (+ 27us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:14:28.117221 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:28.117962 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:28.118916 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:28.119508 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:28.119584 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:28.119593 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:28.119755 (+ 162us) server_negotiation.cc:299] Negotiation successful
0923 23:14:28.119800 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260923 23:14:28.119895 27771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:28.116606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.195:38423 (local address 127.24.227.193:34725)
0923 23:14:28.116801 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:28.116810 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:28.116884 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:28.117082 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:28.117083 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:28.117084 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:14:28.117142 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:28.117144 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:28.118049 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:28.118049 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:28.118842 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:28.118846 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:28.119613 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:28.119614 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:28.119688 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:28.119693 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 23:14:28.119747 (+ 54us) client_negotiation.cc:240] Negotiation successful
0923 23:14:28.119788 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":94,"threads_started":1}
I20260923 23:14:28.120121 27653 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c1449274cde41d5b675dd3c7a39090d" candidate_uuid: "b4a935543f574f50844743bce28cbab2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78106638c5c545219d95183f82117c82" is_pre_election: true
I20260923 23:14:28.120219 27653 raft_consensus.cc:2454] T 9c1449274cde41d5b675dd3c7a39090d P 78106638c5c545219d95183f82117c82 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4a935543f574f50844743bce28cbab2 in term 0.
I20260923 23:14:28.120389 27306 leader_election.cc:302] T 9c1449274cde41d5b675dd3c7a39090d P b4a935543f574f50844743bce28cbab2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 78106638c5c545219d95183f82117c82, b4a935543f574f50844743bce28cbab2; no voters:
I20260923 23:14:28.120523 27728 raft_consensus.cc:2781] T 9c1449274cde41d5b675dd3c7a39090d P b4a935543f574f50844743bce28cbab2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 23:14:28.120563 27728 raft_consensus.cc:482] T 9c1449274cde41d5b675dd3c7a39090d P b4a935543f574f50844743bce28cbab2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 23:14:28.120622 27728 raft_consensus.cc:3037] T 9c1449274cde41d5b675dd3c7a39090d P b4a935543f574f50844743bce28cbab2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:14:28.120716 27721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:28.116716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.193:52645 (local address 127.24.227.194:45731)
0923 23:14:28.116763 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:28.116765 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:28.116865 (+ 100us) server_negotiation.cc:407] Connection header received
0923 23:14:28.116974 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:28.116975 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:28.116984 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:28.117043 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:14:28.118087 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:28.118765 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:28.119712 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:28.120277 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:28.120347 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:28.120355 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:28.120579 (+ 224us) server_negotiation.cc:299] Negotiation successful
0923 23:14:28.120643 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260923 23:14:28.120710 27770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:28.116602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.194:45731 (local address 127.24.227.193:52645)
0923 23:14:28.116793 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:28.116804 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:28.116873 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:28.117047 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:28.117048 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:28.117050 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:14:28.117142 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:28.117145 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:28.118864 (+ 1719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:28.118866 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:28.119625 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:28.119626 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:28.120391 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:28.120392 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:28.120456 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:28.120460 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 23:14:28.120536 (+ 76us) client_negotiation.cc:240] Negotiation successful
0923 23:14:28.120571 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":56,"threads_started":1}
I20260923 23:14:28.120931 27513 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c1449274cde41d5b675dd3c7a39090d" candidate_uuid: "b4a935543f574f50844743bce28cbab2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21a48e797c1942b4bfa349ac15116147" is_pre_election: true
I20260923 23:14:28.121038 27513 raft_consensus.cc:2454] T 9c1449274cde41d5b675dd3c7a39090d P 21a48e797c1942b4bfa349ac15116147 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4a935543f574f50844743bce28cbab2 in term 0.
I20260923 23:14:28.121531 27728 raft_consensus.cc:504] T 9c1449274cde41d5b675dd3c7a39090d P b4a935543f574f50844743bce28cbab2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4a935543f574f50844743bce28cbab2" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 38023 } } peers { permanent_uuid: "21a48e797c1942b4bfa349ac15116147" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 45731 } } peers { permanent_uuid: "78106638c5c545219d95183f82117c82" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 38423 } }
I20260923 23:14:28.121688 27728 leader_election.cc:288] T 9c1449274cde41d5b675dd3c7a39090d P b4a935543f574f50844743bce28cbab2 [CANDIDATE]: Term 1 election: Requested vote from peers 21a48e797c1942b4bfa349ac15116147 (127.24.227.194:45731), 78106638c5c545219d95183f82117c82 (127.24.227.195:38423)
I20260923 23:14:28.121827 27513 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c1449274cde41d5b675dd3c7a39090d" candidate_uuid: "b4a935543f574f50844743bce28cbab2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21a48e797c1942b4bfa349ac15116147"
I20260923 23:14:28.121881 27653 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c1449274cde41d5b675dd3c7a39090d" candidate_uuid: "b4a935543f574f50844743bce28cbab2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78106638c5c545219d95183f82117c82"
I20260923 23:14:28.121919 27513 raft_consensus.cc:3037] T 9c1449274cde41d5b675dd3c7a39090d P 21a48e797c1942b4bfa349ac15116147 [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:14:28.121973 27653 raft_consensus.cc:3037] T 9c1449274cde41d5b675dd3c7a39090d P 78106638c5c545219d95183f82117c82 [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:14:28.122642 27513 raft_consensus.cc:2454] T 9c1449274cde41d5b675dd3c7a39090d P 21a48e797c1942b4bfa349ac15116147 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4a935543f574f50844743bce28cbab2 in term 1.
I20260923 23:14:28.122792 27653 raft_consensus.cc:2454] T 9c1449274cde41d5b675dd3c7a39090d P 78106638c5c545219d95183f82117c82 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4a935543f574f50844743bce28cbab2 in term 1.
I20260923 23:14:28.122817 27307 leader_election.cc:302] T 9c1449274cde41d5b675dd3c7a39090d P b4a935543f574f50844743bce28cbab2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21a48e797c1942b4bfa349ac15116147, b4a935543f574f50844743bce28cbab2; no voters:
I20260923 23:14:28.122922 27728 raft_consensus.cc:2781] T 9c1449274cde41d5b675dd3c7a39090d P b4a935543f574f50844743bce28cbab2 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 23:14:28.123072 27728 raft_consensus.cc:686] T 9c1449274cde41d5b675dd3c7a39090d P b4a935543f574f50844743bce28cbab2 [term 1 LEADER]: Becoming Leader. State: Replica: b4a935543f574f50844743bce28cbab2, State: Running, Role: LEADER
I20260923 23:14:28.123152 27728 consensus_queue.cc:237] T 9c1449274cde41d5b675dd3c7a39090d P b4a935543f574f50844743bce28cbab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4a935543f574f50844743bce28cbab2" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 38023 } } peers { permanent_uuid: "21a48e797c1942b4bfa349ac15116147" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 45731 } } peers { permanent_uuid: "78106638c5c545219d95183f82117c82" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 38423 } }
I20260923 23:14:28.123906 27214 catalog_manager.cc:5495] T 9c1449274cde41d5b675dd3c7a39090d P b4a935543f574f50844743bce28cbab2 reported cstate change: term changed from 0 to 1, leader changed from <none> to b4a935543f574f50844743bce28cbab2 (127.24.227.193). New cstate: current_term: 1 leader_uuid: "b4a935543f574f50844743bce28cbab2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4a935543f574f50844743bce28cbab2" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 38023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "21a48e797c1942b4bfa349ac15116147" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 45731 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "78106638c5c545219d95183f82117c82" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 38423 } health_report { overall_health: UNKNOWN } } }
I20260923 23:14:28.141456 27719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:28.138800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41922 (local address 127.24.227.193:38023)
0923 23:14:28.138844 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:28.138846 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:28.138863 (+ 17us) server_negotiation.cc:407] Connection header received
0923 23:14:28.138898 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:28.138899 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:28.138918 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:28.138998 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 23:14:28.139204 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:28.139934 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:28.140644 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:28.140928 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:28.141008 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:28.141109 (+ 101us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 23:14:28.141246 (+ 137us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 23:14:28.141328 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:28.141348 (+ 20us) server_negotiation.cc:299] Negotiation successful
0923 23:14:28.141370 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260923 23:14:28.146541 27721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:28.144001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.193:49545 (local address 127.24.227.194:45731)
0923 23:14:28.144038 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:28.144040 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:28.144168 (+ 128us) server_negotiation.cc:407] Connection header received
0923 23:14:28.144236 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:28.144237 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:28.144245 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:28.144288 (+ 43us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 23:14:28.144424 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:28.144851 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:28.145882 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:28.146067 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:28.146143 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:28.146334 (+ 191us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 23:14:28.146360 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 23:14:28.146399 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:28.146481 (+ 82us) server_negotiation.cc:299] Negotiation successful
0923 23:14:28.146495 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 23:14:28.146606 27776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:28.143945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.194:45731 (local address 127.24.227.193:49545)
0923 23:14:28.144097 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:28.144108 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:28.144177 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:28.144294 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:28.144295 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:28.144296 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0923 23:14:28.144366 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:28.144367 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:28.144945 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:28.144946 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:28.145799 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:28.145802 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:28.146164 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:28.146165 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:28.146233 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:28.146248 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0923 23:14:28.146421 (+ 173us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0923 23:14:28.146423 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 23:14:28.146477 (+ 54us) client_negotiation.cc:240] Negotiation successful
0923 23:14:28.146517 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":74,"threads_started":1}
I20260923 23:14:28.148597 27513 raft_consensus.cc:1261] T 9c1449274cde41d5b675dd3c7a39090d P 21a48e797c1942b4bfa349ac15116147 [term 1 FOLLOWER]: Refusing update from remote peer b4a935543f574f50844743bce28cbab2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 23:14:28.148592 27653 raft_consensus.cc:1261] T 9c1449274cde41d5b675dd3c7a39090d P 78106638c5c545219d95183f82117c82 [term 1 FOLLOWER]: Refusing update from remote peer b4a935543f574f50844743bce28cbab2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 23:14:28.148928 27772 consensus_queue.cc:1035] T 9c1449274cde41d5b675dd3c7a39090d P b4a935543f574f50844743bce28cbab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "21a48e797c1942b4bfa349ac15116147" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 45731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 23:14:28.149096 27728 consensus_queue.cc:1035] T 9c1449274cde41d5b675dd3c7a39090d P b4a935543f574f50844743bce28cbab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "78106638c5c545219d95183f82117c82" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 38423 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 23:14:28.156352 27565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:28.153908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.193:38023 (local address 127.24.227.194:43411)
0923 23:14:28.153987 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:28.153997 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:28.154032 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:28.154213 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:28.154214 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:28.154215 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0923 23:14:28.154247 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:28.154249 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:28.155030 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:28.155031 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:28.155622 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:28.155625 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:28.156033 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:28.156034 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:28.156099 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:28.156103 (+ 4us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0923 23:14:28.156253 (+ 150us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0923 23:14:28.156254 (+ 1us) client_negotiation.cc:769] Sending connection context
0923 23:14:28.156281 (+ 27us) client_negotiation.cc:240] Negotiation successful
0923 23:14:28.156310 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260923 23:14:28.156354 27719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:28.153948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.194:43411 (local address 127.24.227.193:38023)
0923 23:14:28.154093 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:28.154095 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:28.154101 (+ 6us) server_negotiation.cc:407] Connection header received
0923 23:14:28.154118 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:28.154119 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:28.154128 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:28.154179 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 23:14:28.154310 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:28.154953 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:28.155685 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:28.155961 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:28.156033 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:28.156177 (+ 144us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 23:14:28.156203 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 23:14:28.156253 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:28.156283 (+ 30us) server_negotiation.cc:299] Negotiation successful
0923 23:14:28.156299 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130}
W20260923 23:14:28.156635 27748 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260923 23:14:29.674949 27216 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50660:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 23:14:29.675069 27216 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50660:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 23:14:29.675886 27216 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e6ed2aa900084446a84ca867a45396ae: Sending DeleteTablet for 3 replicas of tablet 9c1449274cde41d5b675dd3c7a39090d
I20260923 23:14:29.676126 27633 tablet_service.cc:1514] Processing DeleteTablet for tablet 9c1449274cde41d5b675dd3c7a39090d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 23:14:29 UTC) from {username='kudu', principal='kudu/127.24.227.254@KRBTEST.COM'} at 127.0.0.1:51296
I20260923 23:14:29.676155 27493 tablet_service.cc:1514] Processing DeleteTablet for tablet 9c1449274cde41d5b675dd3c7a39090d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 23:14:29 UTC) from {username='kudu', principal='kudu/127.24.227.254@KRBTEST.COM'} at 127.0.0.1:43922
I20260923 23:14:29.676146 27353 tablet_service.cc:1514] Processing DeleteTablet for tablet 9c1449274cde41d5b675dd3c7a39090d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 23:14:29 UTC) from {username='kudu', principal='kudu/127.24.227.254@KRBTEST.COM'} at 127.0.0.1:41908
I20260923 23:14:29.676301 27803 tablet_replica.cc:330] T 9c1449274cde41d5b675dd3c7a39090d P 78106638c5c545219d95183f82117c82: stopping tablet replica
I20260923 23:14:29.676337 27805 tablet_replica.cc:330] T 9c1449274cde41d5b675dd3c7a39090d P b4a935543f574f50844743bce28cbab2: stopping tablet replica
I20260923 23:14:29.676348 27803 raft_consensus.cc:2229] T 9c1449274cde41d5b675dd3c7a39090d P 78106638c5c545219d95183f82117c82 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 23:14:29.676379 27803 raft_consensus.cc:2258] T 9c1449274cde41d5b675dd3c7a39090d P 78106638c5c545219d95183f82117c82 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 23:14:29.676384 27805 raft_consensus.cc:2229] T 9c1449274cde41d5b675dd3c7a39090d P b4a935543f574f50844743bce28cbab2 [term 1 LEADER]: Raft consensus shutting down.
I20260923 23:14:29.676462 27804 tablet_replica.cc:330] T 9c1449274cde41d5b675dd3c7a39090d P 21a48e797c1942b4bfa349ac15116147: stopping tablet replica
I20260923 23:14:29.676471 27805 raft_consensus.cc:2258] T 9c1449274cde41d5b675dd3c7a39090d P b4a935543f574f50844743bce28cbab2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 23:14:29.676529 27804 raft_consensus.cc:2229] T 9c1449274cde41d5b675dd3c7a39090d P 21a48e797c1942b4bfa349ac15116147 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 23:14:29.676559 27804 raft_consensus.cc:2258] T 9c1449274cde41d5b675dd3c7a39090d P 21a48e797c1942b4bfa349ac15116147 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 23:14:29.676826 27803 ts_tablet_manager.cc:1905] T 9c1449274cde41d5b675dd3c7a39090d P 78106638c5c545219d95183f82117c82: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 23:14:29.678318 27804 ts_tablet_manager.cc:1905] T 9c1449274cde41d5b675dd3c7a39090d P 21a48e797c1942b4bfa349ac15116147: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 23:14:29.678689 27805 ts_tablet_manager.cc:1905] T 9c1449274cde41d5b675dd3c7a39090d P b4a935543f574f50844743bce28cbab2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 23:14:29.680054 27804 ts_tablet_manager.cc:1918] T 9c1449274cde41d5b675dd3c7a39090d P 21a48e797c1942b4bfa349ac15116147: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260923 23:14:29.680065 27805 ts_tablet_manager.cc:1918] T 9c1449274cde41d5b675dd3c7a39090d P b4a935543f574f50844743bce28cbab2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260923 23:14:29.680123 27804 log.cc:1196] T 9c1449274cde41d5b675dd3c7a39090d P 21a48e797c1942b4bfa349ac15116147: Deleting WAL directory at /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/minicluster-data/ts-1/wal/wals/9c1449274cde41d5b675dd3c7a39090d
I20260923 23:14:29.680229 27803 ts_tablet_manager.cc:1918] T 9c1449274cde41d5b675dd3c7a39090d P 78106638c5c545219d95183f82117c82: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260923 23:14:29.680235 27805 log.cc:1196] T 9c1449274cde41d5b675dd3c7a39090d P b4a935543f574f50844743bce28cbab2: Deleting WAL directory at /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/minicluster-data/ts-0/wal/wals/9c1449274cde41d5b675dd3c7a39090d
I20260923 23:14:29.680289 27803 log.cc:1196] T 9c1449274cde41d5b675dd3c7a39090d P 78106638c5c545219d95183f82117c82: Deleting WAL directory at /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/minicluster-data/ts-2/wal/wals/9c1449274cde41d5b675dd3c7a39090d
I20260923 23:14:29.680361 27804 ts_tablet_manager.cc:1939] T 9c1449274cde41d5b675dd3c7a39090d P 21a48e797c1942b4bfa349ac15116147: Deleting consensus metadata
I20260923 23:14:29.680531 27803 ts_tablet_manager.cc:1939] T 9c1449274cde41d5b675dd3c7a39090d P 78106638c5c545219d95183f82117c82: Deleting consensus metadata
I20260923 23:14:29.680656 27805 ts_tablet_manager.cc:1939] T 9c1449274cde41d5b675dd3c7a39090d P b4a935543f574f50844743bce28cbab2: Deleting consensus metadata
I20260923 23:14:29.680650 27203 catalog_manager.cc:4841] TS 21a48e797c1942b4bfa349ac15116147 (127.24.227.194:45731): tablet 9c1449274cde41d5b675dd3c7a39090d (table test-table [id=8146c81c780c42569bc0f24e9194a9fb]) successfully deleted
I20260923 23:14:29.680871 27202 catalog_manager.cc:4841] TS 78106638c5c545219d95183f82117c82 (127.24.227.195:38423): tablet 9c1449274cde41d5b675dd3c7a39090d (table test-table [id=8146c81c780c42569bc0f24e9194a9fb]) successfully deleted
I20260923 23:14:29.680994 27200 catalog_manager.cc:4841] TS b4a935543f574f50844743bce28cbab2 (127.24.227.193:38023): tablet 9c1449274cde41d5b675dd3c7a39090d (table test-table [id=8146c81c780c42569bc0f24e9194a9fb]) successfully deleted
Sep 23 23:14:29 dist-test-slave-754n krb5kdc[27171](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205269, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 23 23:14:29 dist-test-slave-754n krb5kdc[27171](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205269, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
I20260923 23:14:29.713536 27813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:29.692731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57668 (local address 127.24.227.254:42395)
0923 23:14:29.692851 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:29.692854 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:29.692861 (+ 7us) server_negotiation.cc:407] Connection header received
0923 23:14:29.692932 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:29.692933 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:29.692943 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:29.693021 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:29.693552 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:29.694067 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:29.694796 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:29.694998 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:29.695083 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:29.711321 (+ 16238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:29.711323 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:29.711323 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:29.711329 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:29.712624 (+ 1295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:29.712913 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:29.712914 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:29.712914 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:29.712962 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:29.713119 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:29.713119 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:29.713120 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:29.713217 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:29.713298 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:29.713375 (+ 77us) server_negotiation.cc:299] Negotiation successful
0923 23:14:29.713461 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":65,"threads_started":1}
I20260923 23:14:29.714891 27216 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57668:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 23:14:29.715054 27216 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 23:14:29.716372 27353 tablet_service.cc:1467] Processing CreateTablet for tablet 5d4109e0e8c74a33a5c2bc8e02f5ee6d (DEFAULT_TABLE table=test-table [id=8f16b1fed67d49cda2dc49d639045ea1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 23:14:29.716403 27493 tablet_service.cc:1467] Processing CreateTablet for tablet 5d4109e0e8c74a33a5c2bc8e02f5ee6d (DEFAULT_TABLE table=test-table [id=8f16b1fed67d49cda2dc49d639045ea1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 23:14:29.716392 27633 tablet_service.cc:1467] Processing CreateTablet for tablet 5d4109e0e8c74a33a5c2bc8e02f5ee6d (DEFAULT_TABLE table=test-table [id=8f16b1fed67d49cda2dc49d639045ea1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 23:14:29.716506 27353 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d4109e0e8c74a33a5c2bc8e02f5ee6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:14:29.716514 27493 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d4109e0e8c74a33a5c2bc8e02f5ee6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:14:29.716534 27633 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d4109e0e8c74a33a5c2bc8e02f5ee6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:14:29.717499 27814 tablet_bootstrap.cc:492] T 5d4109e0e8c74a33a5c2bc8e02f5ee6d P 78106638c5c545219d95183f82117c82: Bootstrap starting.
I20260923 23:14:29.717948 27814 tablet_bootstrap.cc:654] T 5d4109e0e8c74a33a5c2bc8e02f5ee6d P 78106638c5c545219d95183f82117c82: Neither blocks nor log segments found. Creating new log.
I20260923 23:14:29.717974 27815 tablet_bootstrap.cc:492] T 5d4109e0e8c74a33a5c2bc8e02f5ee6d P b4a935543f574f50844743bce28cbab2: Bootstrap starting.
I20260923 23:14:29.718360 27815 tablet_bootstrap.cc:654] T 5d4109e0e8c74a33a5c2bc8e02f5ee6d P b4a935543f574f50844743bce28cbab2: Neither blocks nor log segments found. Creating new log.
I20260923 23:14:29.718582 27814 tablet_bootstrap.cc:492] T 5d4109e0e8c74a33a5c2bc8e02f5ee6d P 78106638c5c545219d95183f82117c82: No bootstrap required, opened a new log
I20260923 23:14:29.718650 27816 tablet_bootstrap.cc:492] T 5d4109e0e8c74a33a5c2bc8e02f5ee6d P 21a48e797c1942b4bfa349ac15116147: Bootstrap starting.
I20260923 23:14:29.718652 27814 ts_tablet_manager.cc:1397] T 5d4109e0e8c74a33a5c2bc8e02f5ee6d P 78106638c5c545219d95183f82117c82: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260923 23:14:29.718786 27814 raft_consensus.cc:348] T 5d4109e0e8c74a33a5c2bc8e02f5ee6d P 78106638c5c545219d95183f82117c82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4a935543f574f50844743bce28cbab2" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 38023 } } peers { permanent_uuid: "21a48e797c1942b4bfa349ac15116147" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 45731 } } peers { permanent_uuid: "78106638c5c545219d95183f82117c82" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 38423 } }
I20260923 23:14:29.718870 27814 raft_consensus.cc:374] T 5d4109e0e8c74a33a5c2bc8e02f5ee6d P 78106638c5c545219d95183f82117c82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:14:29.718912 27814 raft_consensus.cc:729] T 5d4109e0e8c74a33a5c2bc8e02f5ee6d P 78106638c5c545219d95183f82117c82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78106638c5c545219d95183f82117c82, State: Initialized, Role: FOLLOWER
I20260923 23:14:29.718976 27816 tablet_bootstrap.cc:654] T 5d4109e0e8c74a33a5c2bc8e02f5ee6d P 21a48e797c1942b4bfa349ac15116147: Neither blocks nor log segments found. Creating new log.
I20260923 23:14:29.718981 27814 consensus_queue.cc:260] T 5d4109e0e8c74a33a5c2bc8e02f5ee6d P 78106638c5c545219d95183f82117c82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4a935543f574f50844743bce28cbab2" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 38023 } } peers { permanent_uuid: "21a48e797c1942b4bfa349ac15116147" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 45731 } } peers { permanent_uuid: "78106638c5c545219d95183f82117c82" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 38423 } }
I20260923 23:14:29.719058 27815 tablet_bootstrap.cc:492] T 5d4109e0e8c74a33a5c2bc8e02f5ee6d P b4a935543f574f50844743bce28cbab2: No bootstrap required, opened a new log
I20260923 23:14:29.719130 27814 ts_tablet_manager.cc:1428] T 5d4109e0e8c74a33a5c2bc8e02f5ee6d P 78106638c5c545219d95183f82117c82: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260923 23:14:29.719144 27815 ts_tablet_manager.cc:1397] T 5d4109e0e8c74a33a5c2bc8e02f5ee6d P b4a935543f574f50844743bce28cbab2: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260923 23:14:29.719300 27815 raft_consensus.cc:348] T 5d4109e0e8c74a33a5c2bc8e02f5ee6d P b4a935543f574f50844743bce28cbab2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4a935543f574f50844743bce28cbab2" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 38023 } } peers { permanent_uuid: "21a48e797c1942b4bfa349ac15116147" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 45731 } } peers { permanent_uuid: "78106638c5c545219d95183f82117c82" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 38423 } }
I20260923 23:14:29.719377 27815 raft_consensus.cc:374] T 5d4109e0e8c74a33a5c2bc8e02f5ee6d P b4a935543f574f50844743bce28cbab2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:14:29.719398 27815 raft_consensus.cc:729] T 5d4109e0e8c74a33a5c2bc8e02f5ee6d P b4a935543f574f50844743bce28cbab2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4a935543f574f50844743bce28cbab2, State: Initialized, Role: FOLLOWER
I20260923 23:14:29.719465 27815 consensus_queue.cc:260] T 5d4109e0e8c74a33a5c2bc8e02f5ee6d P b4a935543f574f50844743bce28cbab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4a935543f574f50844743bce28cbab2" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 38023 } } peers { permanent_uuid: "21a48e797c1942b4bfa349ac15116147" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 45731 } } peers { permanent_uuid: "78106638c5c545219d95183f82117c82" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 38423 } }
I20260923 23:14:29.719550 27816 tablet_bootstrap.cc:492] T 5d4109e0e8c74a33a5c2bc8e02f5ee6d P 21a48e797c1942b4bfa349ac15116147: No bootstrap required, opened a new log
I20260923 23:14:29.719595 27815 ts_tablet_manager.cc:1428] T 5d4109e0e8c74a33a5c2bc8e02f5ee6d P b4a935543f574f50844743bce28cbab2: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:29.719599 27816 ts_tablet_manager.cc:1397] T 5d4109e0e8c74a33a5c2bc8e02f5ee6d P 21a48e797c1942b4bfa349ac15116147: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260923 23:14:29.719708 27816 raft_consensus.cc:348] T 5d4109e0e8c74a33a5c2bc8e02f5ee6d P 21a48e797c1942b4bfa349ac15116147 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4a935543f574f50844743bce28cbab2" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 38023 } } peers { permanent_uuid: "21a48e797c1942b4bfa349ac15116147" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 45731 } } peers { permanent_uuid: "78106638c5c545219d95183f82117c82" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 38423 } }
I20260923 23:14:29.719789 27816 raft_consensus.cc:374] T 5d4109e0e8c74a33a5c2bc8e02f5ee6d P 21a48e797c1942b4bfa349ac15116147 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:14:29.719812 27816 raft_consensus.cc:729] T 5d4109e0e8c74a33a5c2bc8e02f5ee6d P 21a48e797c1942b4bfa349ac15116147 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21a48e797c1942b4bfa349ac15116147, State: Initialized, Role: FOLLOWER
I20260923 23:14:29.719908 27816 consensus_queue.cc:260] T 5d4109e0e8c74a33a5c2bc8e02f5ee6d P 21a48e797c1942b4bfa349ac15116147 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4a935543f574f50844743bce28cbab2" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 38023 } } peers { permanent_uuid: "21a48e797c1942b4bfa349ac15116147" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 45731 } } peers { permanent_uuid: "78106638c5c545219d95183f82117c82" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 38423 } }
I20260923 23:14:29.720069 27816 ts_tablet_manager.cc:1428] T 5d4109e0e8c74a33a5c2bc8e02f5ee6d P 21a48e797c1942b4bfa349ac15116147: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260923 23:14:29.944518 27759 raft_consensus.cc:482] T 5d4109e0e8c74a33a5c2bc8e02f5ee6d P 21a48e797c1942b4bfa349ac15116147 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 23:14:29.944620 27759 raft_consensus.cc:504] T 5d4109e0e8c74a33a5c2bc8e02f5ee6d P 21a48e797c1942b4bfa349ac15116147 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4a935543f574f50844743bce28cbab2" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 38023 } } peers { permanent_uuid: "21a48e797c1942b4bfa349ac15116147" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 45731 } } peers { permanent_uuid: "78106638c5c545219d95183f82117c82" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 38423 } }
I20260923 23:14:29.944792 27759 leader_election.cc:288] T 5d4109e0e8c74a33a5c2bc8e02f5ee6d P 21a48e797c1942b4bfa349ac15116147 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b4a935543f574f50844743bce28cbab2 (127.24.227.193:38023), 78106638c5c545219d95183f82117c82 (127.24.227.195:38423)
I20260923 23:14:29.944995 27373 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d4109e0e8c74a33a5c2bc8e02f5ee6d" candidate_uuid: "21a48e797c1942b4bfa349ac15116147" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4a935543f574f50844743bce28cbab2" is_pre_election: true
I20260923 23:14:29.945108 27373 raft_consensus.cc:2454] T 5d4109e0e8c74a33a5c2bc8e02f5ee6d P b4a935543f574f50844743bce28cbab2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 21a48e797c1942b4bfa349ac15116147 in term 0.
I20260923 23:14:29.945041 27652 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d4109e0e8c74a33a5c2bc8e02f5ee6d" candidate_uuid: "21a48e797c1942b4bfa349ac15116147" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78106638c5c545219d95183f82117c82" is_pre_election: true
I20260923 23:14:29.945166 27652 raft_consensus.cc:2454] T 5d4109e0e8c74a33a5c2bc8e02f5ee6d P 78106638c5c545219d95183f82117c82 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 21a48e797c1942b4bfa349ac15116147 in term 0.
I20260923 23:14:29.945283 27444 leader_election.cc:302] T 5d4109e0e8c74a33a5c2bc8e02f5ee6d P 21a48e797c1942b4bfa349ac15116147 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21a48e797c1942b4bfa349ac15116147, b4a935543f574f50844743bce28cbab2; no voters:
I20260923 23:14:29.945420 27759 raft_consensus.cc:2781] T 5d4109e0e8c74a33a5c2bc8e02f5ee6d P 21a48e797c1942b4bfa349ac15116147 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 23:14:29.945472 27759 raft_consensus.cc:482] T 5d4109e0e8c74a33a5c2bc8e02f5ee6d P 21a48e797c1942b4bfa349ac15116147 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 23:14:29.945487 27759 raft_consensus.cc:3037] T 5d4109e0e8c74a33a5c2bc8e02f5ee6d P 21a48e797c1942b4bfa349ac15116147 [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:14:29.946147 27759 raft_consensus.cc:504] T 5d4109e0e8c74a33a5c2bc8e02f5ee6d P 21a48e797c1942b4bfa349ac15116147 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4a935543f574f50844743bce28cbab2" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 38023 } } peers { permanent_uuid: "21a48e797c1942b4bfa349ac15116147" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 45731 } } peers { permanent_uuid: "78106638c5c545219d95183f82117c82" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 38423 } }
I20260923 23:14:29.946274 27759 leader_election.cc:288] T 5d4109e0e8c74a33a5c2bc8e02f5ee6d P 21a48e797c1942b4bfa349ac15116147 [CANDIDATE]: Term 1 election: Requested vote from peers b4a935543f574f50844743bce28cbab2 (127.24.227.193:38023), 78106638c5c545219d95183f82117c82 (127.24.227.195:38423)
I20260923 23:14:29.946410 27373 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d4109e0e8c74a33a5c2bc8e02f5ee6d" candidate_uuid: "21a48e797c1942b4bfa349ac15116147" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4a935543f574f50844743bce28cbab2"
I20260923 23:14:29.946420 27652 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d4109e0e8c74a33a5c2bc8e02f5ee6d" candidate_uuid: "21a48e797c1942b4bfa349ac15116147" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78106638c5c545219d95183f82117c82"
I20260923 23:14:29.946494 27373 raft_consensus.cc:3037] T 5d4109e0e8c74a33a5c2bc8e02f5ee6d P b4a935543f574f50844743bce28cbab2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:14:29.946516 27652 raft_consensus.cc:3037] T 5d4109e0e8c74a33a5c2bc8e02f5ee6d P 78106638c5c545219d95183f82117c82 [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:14:29.947075 27373 raft_consensus.cc:2454] T 5d4109e0e8c74a33a5c2bc8e02f5ee6d P b4a935543f574f50844743bce28cbab2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21a48e797c1942b4bfa349ac15116147 in term 1.
I20260923 23:14:29.947245 27444 leader_election.cc:302] T 5d4109e0e8c74a33a5c2bc8e02f5ee6d P 21a48e797c1942b4bfa349ac15116147 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21a48e797c1942b4bfa349ac15116147, b4a935543f574f50844743bce28cbab2; no voters:
I20260923 23:14:29.947266 27652 raft_consensus.cc:2454] T 5d4109e0e8c74a33a5c2bc8e02f5ee6d P 78106638c5c545219d95183f82117c82 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21a48e797c1942b4bfa349ac15116147 in term 1.
I20260923 23:14:29.947350 27759 raft_consensus.cc:2781] T 5d4109e0e8c74a33a5c2bc8e02f5ee6d P 21a48e797c1942b4bfa349ac15116147 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 23:14:29.947418 27759 raft_consensus.cc:686] T 5d4109e0e8c74a33a5c2bc8e02f5ee6d P 21a48e797c1942b4bfa349ac15116147 [term 1 LEADER]: Becoming Leader. State: Replica: 21a48e797c1942b4bfa349ac15116147, State: Running, Role: LEADER
I20260923 23:14:29.947464 27759 consensus_queue.cc:237] T 5d4109e0e8c74a33a5c2bc8e02f5ee6d P 21a48e797c1942b4bfa349ac15116147 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4a935543f574f50844743bce28cbab2" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 38023 } } peers { permanent_uuid: "21a48e797c1942b4bfa349ac15116147" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 45731 } } peers { permanent_uuid: "78106638c5c545219d95183f82117c82" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 38423 } }
I20260923 23:14:29.948166 27216 catalog_manager.cc:5495] T 5d4109e0e8c74a33a5c2bc8e02f5ee6d P 21a48e797c1942b4bfa349ac15116147 reported cstate change: term changed from 0 to 1, leader changed from <none> to 21a48e797c1942b4bfa349ac15116147 (127.24.227.194). New cstate: current_term: 1 leader_uuid: "21a48e797c1942b4bfa349ac15116147" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4a935543f574f50844743bce28cbab2" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 38023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "21a48e797c1942b4bfa349ac15116147" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 45731 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "78106638c5c545219d95183f82117c82" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 38423 } health_report { overall_health: UNKNOWN } } }
I20260923 23:14:30.003429 27823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:30.001279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56064 (local address 127.24.227.194:45731)
0923 23:14:30.001432 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:30.001434 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:30.001442 (+ 8us) server_negotiation.cc:407] Connection header received
0923 23:14:30.001471 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:30.001471 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:30.001480 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:30.001541 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 23:14:30.001653 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:30.002123 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:30.002826 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:30.003008 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:30.003055 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:30.003180 (+ 125us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 23:14:30.003205 (+ 25us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 23:14:30.003242 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:30.003295 (+ 53us) server_negotiation.cc:299] Negotiation successful
0923 23:14:30.003312 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":99,"threads_started":1}
I20260923 23:14:30.005417 27373 raft_consensus.cc:1261] T 5d4109e0e8c74a33a5c2bc8e02f5ee6d P b4a935543f574f50844743bce28cbab2 [term 1 FOLLOWER]: Refusing update from remote peer 21a48e797c1942b4bfa349ac15116147: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 23:14:30.005506 27652 raft_consensus.cc:1261] T 5d4109e0e8c74a33a5c2bc8e02f5ee6d P 78106638c5c545219d95183f82117c82 [term 1 FOLLOWER]: Refusing update from remote peer 21a48e797c1942b4bfa349ac15116147: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 23:14:30.005627 27789 consensus_queue.cc:1035] T 5d4109e0e8c74a33a5c2bc8e02f5ee6d P 21a48e797c1942b4bfa349ac15116147 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4a935543f574f50844743bce28cbab2" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 38023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 23:14:30.005724 27784 consensus_queue.cc:1035] T 5d4109e0e8c74a33a5c2bc8e02f5ee6d P 21a48e797c1942b4bfa349ac15116147 [LEADER]: Connected to new peer: Peer: permanent_uuid: "78106638c5c545219d95183f82117c82" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 38423 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 23:14:30.016157 27215 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57668:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 23:14:30.016260 27215 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57668:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 23:14:30.016852 27215 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e6ed2aa900084446a84ca867a45396ae: Sending DeleteTablet for 3 replicas of tablet 5d4109e0e8c74a33a5c2bc8e02f5ee6d
I20260923 23:14:30.017077 27492 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d4109e0e8c74a33a5c2bc8e02f5ee6d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 23:14:30 UTC) from {username='kudu', principal='kudu/127.24.227.254@KRBTEST.COM'} at 127.0.0.1:43922
I20260923 23:14:30.017123 27353 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d4109e0e8c74a33a5c2bc8e02f5ee6d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 23:14:30 UTC) from {username='kudu', principal='kudu/127.24.227.254@KRBTEST.COM'} at 127.0.0.1:41908
I20260923 23:14:30.017171 27804 tablet_replica.cc:330] T 5d4109e0e8c74a33a5c2bc8e02f5ee6d P 21a48e797c1942b4bfa349ac15116147: stopping tablet replica
I20260923 23:14:30.017228 27804 raft_consensus.cc:2229] T 5d4109e0e8c74a33a5c2bc8e02f5ee6d P 21a48e797c1942b4bfa349ac15116147 [term 1 LEADER]: Raft consensus shutting down.
I20260923 23:14:30.017283 27804 raft_consensus.cc:2258] T 5d4109e0e8c74a33a5c2bc8e02f5ee6d P 21a48e797c1942b4bfa349ac15116147 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 23:14:30.017292 27215 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57668:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260923 23:14:30.017391 27805 tablet_replica.cc:330] T 5d4109e0e8c74a33a5c2bc8e02f5ee6d P b4a935543f574f50844743bce28cbab2: stopping tablet replica
I20260923 23:14:30.017452 27805 raft_consensus.cc:2229] T 5d4109e0e8c74a33a5c2bc8e02f5ee6d P b4a935543f574f50844743bce28cbab2 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260923 23:14:30.017452 27215 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 23:14:30.017480 27805 raft_consensus.cc:2258] T 5d4109e0e8c74a33a5c2bc8e02f5ee6d P b4a935543f574f50844743bce28cbab2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 23:14:30.017593 27804 ts_tablet_manager.cc:1905] T 5d4109e0e8c74a33a5c2bc8e02f5ee6d P 21a48e797c1942b4bfa349ac15116147: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 23:14:30.017642 27633 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d4109e0e8c74a33a5c2bc8e02f5ee6d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 23:14:30 UTC) from {username='kudu', principal='kudu/127.24.227.254@KRBTEST.COM'} at 127.0.0.1:51296
I20260923 23:14:30.017719 27803 tablet_replica.cc:330] T 5d4109e0e8c74a33a5c2bc8e02f5ee6d P 78106638c5c545219d95183f82117c82: stopping tablet replica
I20260923 23:14:30.017757 27803 raft_consensus.cc:2229] T 5d4109e0e8c74a33a5c2bc8e02f5ee6d P 78106638c5c545219d95183f82117c82 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 23:14:30.017791 27803 raft_consensus.cc:2258] T 5d4109e0e8c74a33a5c2bc8e02f5ee6d P 78106638c5c545219d95183f82117c82 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 23:14:30.017781 27805 ts_tablet_manager.cc:1905] T 5d4109e0e8c74a33a5c2bc8e02f5ee6d P b4a935543f574f50844743bce28cbab2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 23:14:30.019052 27803 ts_tablet_manager.cc:1905] T 5d4109e0e8c74a33a5c2bc8e02f5ee6d P 78106638c5c545219d95183f82117c82: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 23:14:30.019243 27804 ts_tablet_manager.cc:1918] T 5d4109e0e8c74a33a5c2bc8e02f5ee6d P 21a48e797c1942b4bfa349ac15116147: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 23:14:30.019249 27805 ts_tablet_manager.cc:1918] T 5d4109e0e8c74a33a5c2bc8e02f5ee6d P b4a935543f574f50844743bce28cbab2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 23:14:30.019297 27804 log.cc:1196] T 5d4109e0e8c74a33a5c2bc8e02f5ee6d P 21a48e797c1942b4bfa349ac15116147: Deleting WAL directory at /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/minicluster-data/ts-1/wal/wals/5d4109e0e8c74a33a5c2bc8e02f5ee6d
I20260923 23:14:30.019297 27805 log.cc:1196] T 5d4109e0e8c74a33a5c2bc8e02f5ee6d P b4a935543f574f50844743bce28cbab2: Deleting WAL directory at /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/minicluster-data/ts-0/wal/wals/5d4109e0e8c74a33a5c2bc8e02f5ee6d
I20260923 23:14:30.019547 27804 ts_tablet_manager.cc:1939] T 5d4109e0e8c74a33a5c2bc8e02f5ee6d P 21a48e797c1942b4bfa349ac15116147: Deleting consensus metadata
I20260923 23:14:30.019553 27805 ts_tablet_manager.cc:1939] T 5d4109e0e8c74a33a5c2bc8e02f5ee6d P b4a935543f574f50844743bce28cbab2: Deleting consensus metadata
I20260923 23:14:30.019942 27200 catalog_manager.cc:4841] TS b4a935543f574f50844743bce28cbab2 (127.24.227.193:38023): tablet 5d4109e0e8c74a33a5c2bc8e02f5ee6d (table test-table [id=8f16b1fed67d49cda2dc49d639045ea1]) successfully deleted
I20260923 23:14:30.019914 27203 catalog_manager.cc:4841] TS 21a48e797c1942b4bfa349ac15116147 (127.24.227.194:45731): tablet 5d4109e0e8c74a33a5c2bc8e02f5ee6d (table test-table [id=8f16b1fed67d49cda2dc49d639045ea1]) successfully deleted
I20260923 23:14:30.020129 27353 tablet_service.cc:1467] Processing CreateTablet for tablet cbac84d66d5a46bbbcf2bacbae937567 (DEFAULT_TABLE table=test-table [id=15b5d78d4ec94c8ba14aed97ac993ff5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 23:14:30.020141 27633 tablet_service.cc:1467] Processing CreateTablet for tablet cbac84d66d5a46bbbcf2bacbae937567 (DEFAULT_TABLE table=test-table [id=15b5d78d4ec94c8ba14aed97ac993ff5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 23:14:30.020229 27633 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cbac84d66d5a46bbbcf2bacbae937567. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:14:30.020229 27353 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cbac84d66d5a46bbbcf2bacbae937567. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:14:30.020242 27492 tablet_service.cc:1467] Processing CreateTablet for tablet cbac84d66d5a46bbbcf2bacbae937567 (DEFAULT_TABLE table=test-table [id=15b5d78d4ec94c8ba14aed97ac993ff5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 23:14:30.020324 27492 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cbac84d66d5a46bbbcf2bacbae937567. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:14:30.020704 27803 ts_tablet_manager.cc:1918] T 5d4109e0e8c74a33a5c2bc8e02f5ee6d P 78106638c5c545219d95183f82117c82: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 23:14:30.020756 27803 log.cc:1196] T 5d4109e0e8c74a33a5c2bc8e02f5ee6d P 78106638c5c545219d95183f82117c82: Deleting WAL directory at /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/minicluster-data/ts-2/wal/wals/5d4109e0e8c74a33a5c2bc8e02f5ee6d
I20260923 23:14:30.020960 27803 ts_tablet_manager.cc:1939] T 5d4109e0e8c74a33a5c2bc8e02f5ee6d P 78106638c5c545219d95183f82117c82: Deleting consensus metadata
I20260923 23:14:30.021358 27202 catalog_manager.cc:4841] TS 78106638c5c545219d95183f82117c82 (127.24.227.195:38423): tablet 5d4109e0e8c74a33a5c2bc8e02f5ee6d (table test-table [id=8f16b1fed67d49cda2dc49d639045ea1]) successfully deleted
I20260923 23:14:30.021425 27815 tablet_bootstrap.cc:492] T cbac84d66d5a46bbbcf2bacbae937567 P b4a935543f574f50844743bce28cbab2: Bootstrap starting.
I20260923 23:14:30.021631 27816 tablet_bootstrap.cc:492] T cbac84d66d5a46bbbcf2bacbae937567 P 21a48e797c1942b4bfa349ac15116147: Bootstrap starting.
I20260923 23:14:30.021633 27814 tablet_bootstrap.cc:492] T cbac84d66d5a46bbbcf2bacbae937567 P 78106638c5c545219d95183f82117c82: Bootstrap starting.
I20260923 23:14:30.021723 27815 tablet_bootstrap.cc:654] T cbac84d66d5a46bbbcf2bacbae937567 P b4a935543f574f50844743bce28cbab2: Neither blocks nor log segments found. Creating new log.
I20260923 23:14:30.021862 27816 tablet_bootstrap.cc:654] T cbac84d66d5a46bbbcf2bacbae937567 P 21a48e797c1942b4bfa349ac15116147: Neither blocks nor log segments found. Creating new log.
I20260923 23:14:30.022035 27814 tablet_bootstrap.cc:654] T cbac84d66d5a46bbbcf2bacbae937567 P 78106638c5c545219d95183f82117c82: Neither blocks nor log segments found. Creating new log.
I20260923 23:14:30.022331 27815 tablet_bootstrap.cc:492] T cbac84d66d5a46bbbcf2bacbae937567 P b4a935543f574f50844743bce28cbab2: No bootstrap required, opened a new log
I20260923 23:14:30.022385 27815 ts_tablet_manager.cc:1397] T cbac84d66d5a46bbbcf2bacbae937567 P b4a935543f574f50844743bce28cbab2: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260923 23:14:30.022522 27815 raft_consensus.cc:348] T cbac84d66d5a46bbbcf2bacbae937567 P b4a935543f574f50844743bce28cbab2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4a935543f574f50844743bce28cbab2" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 38023 } } peers { permanent_uuid: "21a48e797c1942b4bfa349ac15116147" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 45731 } } peers { permanent_uuid: "78106638c5c545219d95183f82117c82" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 38423 } }
I20260923 23:14:30.022614 27814 tablet_bootstrap.cc:492] T cbac84d66d5a46bbbcf2bacbae937567 P 78106638c5c545219d95183f82117c82: No bootstrap required, opened a new log
I20260923 23:14:30.022626 27815 raft_consensus.cc:374] T cbac84d66d5a46bbbcf2bacbae937567 P b4a935543f574f50844743bce28cbab2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:14:30.022650 27815 raft_consensus.cc:729] T cbac84d66d5a46bbbcf2bacbae937567 P b4a935543f574f50844743bce28cbab2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4a935543f574f50844743bce28cbab2, State: Initialized, Role: FOLLOWER
I20260923 23:14:30.022655 27814 ts_tablet_manager.cc:1397] T cbac84d66d5a46bbbcf2bacbae937567 P 78106638c5c545219d95183f82117c82: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260923 23:14:30.022704 27815 consensus_queue.cc:260] T cbac84d66d5a46bbbcf2bacbae937567 P b4a935543f574f50844743bce28cbab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4a935543f574f50844743bce28cbab2" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 38023 } } peers { permanent_uuid: "21a48e797c1942b4bfa349ac15116147" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 45731 } } peers { permanent_uuid: "78106638c5c545219d95183f82117c82" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 38423 } }
I20260923 23:14:30.022801 27815 ts_tablet_manager.cc:1428] T cbac84d66d5a46bbbcf2bacbae937567 P b4a935543f574f50844743bce28cbab2: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260923 23:14:30.022783 27814 raft_consensus.cc:348] T cbac84d66d5a46bbbcf2bacbae937567 P 78106638c5c545219d95183f82117c82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4a935543f574f50844743bce28cbab2" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 38023 } } peers { permanent_uuid: "21a48e797c1942b4bfa349ac15116147" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 45731 } } peers { permanent_uuid: "78106638c5c545219d95183f82117c82" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 38423 } }
I20260923 23:14:30.022853 27814 raft_consensus.cc:374] T cbac84d66d5a46bbbcf2bacbae937567 P 78106638c5c545219d95183f82117c82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:14:30.022884 27814 raft_consensus.cc:729] T cbac84d66d5a46bbbcf2bacbae937567 P 78106638c5c545219d95183f82117c82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78106638c5c545219d95183f82117c82, State: Initialized, Role: FOLLOWER
I20260923 23:14:30.022928 27814 consensus_queue.cc:260] T cbac84d66d5a46bbbcf2bacbae937567 P 78106638c5c545219d95183f82117c82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4a935543f574f50844743bce28cbab2" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 38023 } } peers { permanent_uuid: "21a48e797c1942b4bfa349ac15116147" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 45731 } } peers { permanent_uuid: "78106638c5c545219d95183f82117c82" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 38423 } }
I20260923 23:14:30.023016 27814 ts_tablet_manager.cc:1428] T cbac84d66d5a46bbbcf2bacbae937567 P 78106638c5c545219d95183f82117c82: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:30.023567 27816 tablet_bootstrap.cc:492] T cbac84d66d5a46bbbcf2bacbae937567 P 21a48e797c1942b4bfa349ac15116147: No bootstrap required, opened a new log
I20260923 23:14:30.023618 27816 ts_tablet_manager.cc:1397] T cbac84d66d5a46bbbcf2bacbae937567 P 21a48e797c1942b4bfa349ac15116147: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20260923 23:14:30.023727 27816 raft_consensus.cc:348] T cbac84d66d5a46bbbcf2bacbae937567 P 21a48e797c1942b4bfa349ac15116147 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4a935543f574f50844743bce28cbab2" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 38023 } } peers { permanent_uuid: "21a48e797c1942b4bfa349ac15116147" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 45731 } } peers { permanent_uuid: "78106638c5c545219d95183f82117c82" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 38423 } }
I20260923 23:14:30.023785 27816 raft_consensus.cc:374] T cbac84d66d5a46bbbcf2bacbae937567 P 21a48e797c1942b4bfa349ac15116147 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:14:30.023799 27816 raft_consensus.cc:729] T cbac84d66d5a46bbbcf2bacbae937567 P 21a48e797c1942b4bfa349ac15116147 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21a48e797c1942b4bfa349ac15116147, State: Initialized, Role: FOLLOWER
I20260923 23:14:30.023844 27816 consensus_queue.cc:260] T cbac84d66d5a46bbbcf2bacbae937567 P 21a48e797c1942b4bfa349ac15116147 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4a935543f574f50844743bce28cbab2" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 38023 } } peers { permanent_uuid: "21a48e797c1942b4bfa349ac15116147" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 45731 } } peers { permanent_uuid: "78106638c5c545219d95183f82117c82" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 38423 } }
I20260923 23:14:30.023947 27816 ts_tablet_manager.cc:1428] T cbac84d66d5a46bbbcf2bacbae937567 P 21a48e797c1942b4bfa349ac15116147: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:30.108364 27759 raft_consensus.cc:482] T cbac84d66d5a46bbbcf2bacbae937567 P 21a48e797c1942b4bfa349ac15116147 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 23:14:30.108469 27759 raft_consensus.cc:504] T cbac84d66d5a46bbbcf2bacbae937567 P 21a48e797c1942b4bfa349ac15116147 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4a935543f574f50844743bce28cbab2" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 38023 } } peers { permanent_uuid: "21a48e797c1942b4bfa349ac15116147" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 45731 } } peers { permanent_uuid: "78106638c5c545219d95183f82117c82" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 38423 } }
I20260923 23:14:30.108688 27759 leader_election.cc:288] T cbac84d66d5a46bbbcf2bacbae937567 P 21a48e797c1942b4bfa349ac15116147 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b4a935543f574f50844743bce28cbab2 (127.24.227.193:38023), 78106638c5c545219d95183f82117c82 (127.24.227.195:38423)
I20260923 23:14:30.108848 27373 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cbac84d66d5a46bbbcf2bacbae937567" candidate_uuid: "21a48e797c1942b4bfa349ac15116147" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4a935543f574f50844743bce28cbab2" is_pre_election: true
I20260923 23:14:30.108940 27373 raft_consensus.cc:2454] T cbac84d66d5a46bbbcf2bacbae937567 P b4a935543f574f50844743bce28cbab2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 21a48e797c1942b4bfa349ac15116147 in term 0.
I20260923 23:14:30.108881 27652 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cbac84d66d5a46bbbcf2bacbae937567" candidate_uuid: "21a48e797c1942b4bfa349ac15116147" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78106638c5c545219d95183f82117c82" is_pre_election: true
I20260923 23:14:30.108999 27652 raft_consensus.cc:2454] T cbac84d66d5a46bbbcf2bacbae937567 P 78106638c5c545219d95183f82117c82 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 21a48e797c1942b4bfa349ac15116147 in term 0.
I20260923 23:14:30.109167 27444 leader_election.cc:302] T cbac84d66d5a46bbbcf2bacbae937567 P 21a48e797c1942b4bfa349ac15116147 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21a48e797c1942b4bfa349ac15116147, b4a935543f574f50844743bce28cbab2; no voters:
I20260923 23:14:30.109278 27759 raft_consensus.cc:2781] T cbac84d66d5a46bbbcf2bacbae937567 P 21a48e797c1942b4bfa349ac15116147 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 23:14:30.109315 27759 raft_consensus.cc:482] T cbac84d66d5a46bbbcf2bacbae937567 P 21a48e797c1942b4bfa349ac15116147 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 23:14:30.109340 27759 raft_consensus.cc:3037] T cbac84d66d5a46bbbcf2bacbae937567 P 21a48e797c1942b4bfa349ac15116147 [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:14:30.110056 27759 raft_consensus.cc:504] T cbac84d66d5a46bbbcf2bacbae937567 P 21a48e797c1942b4bfa349ac15116147 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4a935543f574f50844743bce28cbab2" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 38023 } } peers { permanent_uuid: "21a48e797c1942b4bfa349ac15116147" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 45731 } } peers { permanent_uuid: "78106638c5c545219d95183f82117c82" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 38423 } }
I20260923 23:14:30.110194 27759 leader_election.cc:288] T cbac84d66d5a46bbbcf2bacbae937567 P 21a48e797c1942b4bfa349ac15116147 [CANDIDATE]: Term 1 election: Requested vote from peers b4a935543f574f50844743bce28cbab2 (127.24.227.193:38023), 78106638c5c545219d95183f82117c82 (127.24.227.195:38423)
I20260923 23:14:30.110352 27373 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cbac84d66d5a46bbbcf2bacbae937567" candidate_uuid: "21a48e797c1942b4bfa349ac15116147" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4a935543f574f50844743bce28cbab2"
I20260923 23:14:30.110447 27373 raft_consensus.cc:3037] T cbac84d66d5a46bbbcf2bacbae937567 P b4a935543f574f50844743bce28cbab2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:14:30.110425 27652 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cbac84d66d5a46bbbcf2bacbae937567" candidate_uuid: "21a48e797c1942b4bfa349ac15116147" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78106638c5c545219d95183f82117c82"
I20260923 23:14:30.110481 27652 raft_consensus.cc:3037] T cbac84d66d5a46bbbcf2bacbae937567 P 78106638c5c545219d95183f82117c82 [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:14:30.111210 27373 raft_consensus.cc:2454] T cbac84d66d5a46bbbcf2bacbae937567 P b4a935543f574f50844743bce28cbab2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21a48e797c1942b4bfa349ac15116147 in term 1.
I20260923 23:14:30.111209 27652 raft_consensus.cc:2454] T cbac84d66d5a46bbbcf2bacbae937567 P 78106638c5c545219d95183f82117c82 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21a48e797c1942b4bfa349ac15116147 in term 1.
I20260923 23:14:30.111387 27446 leader_election.cc:302] T cbac84d66d5a46bbbcf2bacbae937567 P 21a48e797c1942b4bfa349ac15116147 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21a48e797c1942b4bfa349ac15116147, 78106638c5c545219d95183f82117c82; no voters:
I20260923 23:14:30.111495 27759 raft_consensus.cc:2781] T cbac84d66d5a46bbbcf2bacbae937567 P 21a48e797c1942b4bfa349ac15116147 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 23:14:30.111577 27759 raft_consensus.cc:686] T cbac84d66d5a46bbbcf2bacbae937567 P 21a48e797c1942b4bfa349ac15116147 [term 1 LEADER]: Becoming Leader. State: Replica: 21a48e797c1942b4bfa349ac15116147, State: Running, Role: LEADER
I20260923 23:14:30.111637 27759 consensus_queue.cc:237] T cbac84d66d5a46bbbcf2bacbae937567 P 21a48e797c1942b4bfa349ac15116147 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4a935543f574f50844743bce28cbab2" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 38023 } } peers { permanent_uuid: "21a48e797c1942b4bfa349ac15116147" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 45731 } } peers { permanent_uuid: "78106638c5c545219d95183f82117c82" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 38423 } }
I20260923 23:14:30.112203 27215 catalog_manager.cc:5495] T cbac84d66d5a46bbbcf2bacbae937567 P 21a48e797c1942b4bfa349ac15116147 reported cstate change: term changed from 0 to 1, leader changed from <none> to 21a48e797c1942b4bfa349ac15116147 (127.24.227.194). New cstate: current_term: 1 leader_uuid: "21a48e797c1942b4bfa349ac15116147" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4a935543f574f50844743bce28cbab2" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 38023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "21a48e797c1942b4bfa349ac15116147" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 45731 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "78106638c5c545219d95183f82117c82" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 38423 } health_report { overall_health: UNKNOWN } } }
I20260923 23:14:30.118954 27652 raft_consensus.cc:1261] T cbac84d66d5a46bbbcf2bacbae937567 P 78106638c5c545219d95183f82117c82 [term 1 FOLLOWER]: Refusing update from remote peer 21a48e797c1942b4bfa349ac15116147: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 23:14:30.118963 27373 raft_consensus.cc:1261] T cbac84d66d5a46bbbcf2bacbae937567 P b4a935543f574f50844743bce28cbab2 [term 1 FOLLOWER]: Refusing update from remote peer 21a48e797c1942b4bfa349ac15116147: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 23:14:30.119153 27784 consensus_queue.cc:1035] T cbac84d66d5a46bbbcf2bacbae937567 P 21a48e797c1942b4bfa349ac15116147 [LEADER]: Connected to new peer: Peer: permanent_uuid: "78106638c5c545219d95183f82117c82" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 38423 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 23:14:30.119252 27789 consensus_queue.cc:1035] T cbac84d66d5a46bbbcf2bacbae937567 P 21a48e797c1942b4bfa349ac15116147 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4a935543f574f50844743bce28cbab2" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 38023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260923 23:14:30.122977 27782 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260923 23:14:31.641026 27215 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57668:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 23:14:31.641122 27215 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57668:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 23:14:31.641832 27215 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e6ed2aa900084446a84ca867a45396ae: Sending DeleteTablet for 3 replicas of tablet cbac84d66d5a46bbbcf2bacbae937567
I20260923 23:14:31.642107 27493 tablet_service.cc:1514] Processing DeleteTablet for tablet cbac84d66d5a46bbbcf2bacbae937567 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 23:14:31 UTC) from {username='kudu', principal='kudu/127.24.227.254@KRBTEST.COM'} at 127.0.0.1:43922
I20260923 23:14:31.642103 27633 tablet_service.cc:1514] Processing DeleteTablet for tablet cbac84d66d5a46bbbcf2bacbae937567 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 23:14:31 UTC) from {username='kudu', principal='kudu/127.24.227.254@KRBTEST.COM'} at 127.0.0.1:51296
I20260923 23:14:31.642197 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 27288
I20260923 23:14:31.642318 27353 tablet_service.cc:1514] Processing DeleteTablet for tablet cbac84d66d5a46bbbcf2bacbae937567 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 23:14:31 UTC) from {username='kudu', principal='kudu/127.24.227.254@KRBTEST.COM'} at 127.0.0.1:41908
I20260923 23:14:31.643262 27853 tablet_replica.cc:330] T cbac84d66d5a46bbbcf2bacbae937567 P 78106638c5c545219d95183f82117c82: stopping tablet replica
I20260923 23:14:31.643307 27853 raft_consensus.cc:2229] T cbac84d66d5a46bbbcf2bacbae937567 P 78106638c5c545219d95183f82117c82 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 23:14:31.643332 27853 raft_consensus.cc:2258] T cbac84d66d5a46bbbcf2bacbae937567 P 78106638c5c545219d95183f82117c82 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 23:14:31.643628 27853 ts_tablet_manager.cc:1905] T cbac84d66d5a46bbbcf2bacbae937567 P 78106638c5c545219d95183f82117c82: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 23:14:31.645524 27853 ts_tablet_manager.cc:1918] T cbac84d66d5a46bbbcf2bacbae937567 P 78106638c5c545219d95183f82117c82: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260923 23:14:31.645576 27853 log.cc:1196] T cbac84d66d5a46bbbcf2bacbae937567 P 78106638c5c545219d95183f82117c82: Deleting WAL directory at /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790205259943518-25487-0/minicluster-data/ts-2/wal/wals/cbac84d66d5a46bbbcf2bacbae937567
I20260923 23:14:31.645797 27853 ts_tablet_manager.cc:1939] T cbac84d66d5a46bbbcf2bacbae937567 P 78106638c5c545219d95183f82117c82: Deleting consensus metadata
I20260923 23:14:31.646126 27202 catalog_manager.cc:4841] TS 78106638c5c545219d95183f82117c82 (127.24.227.195:38423): tablet cbac84d66d5a46bbbcf2bacbae937567 (table test-table [id=15b5d78d4ec94c8ba14aed97ac993ff5]) successfully deleted
I20260923 23:14:31.647985 27852 tablet_replica.cc:330] T cbac84d66d5a46bbbcf2bacbae937567 P 21a48e797c1942b4bfa349ac15116147: stopping tablet replica
I20260923 23:14:31.648056 27852 raft_consensus.cc:2229] T cbac84d66d5a46bbbcf2bacbae937567 P 21a48e797c1942b4bfa349ac15116147 [term 1 LEADER]: Raft consensus shutting down.
I20260923 23:14:31.648159 27852 raft_consensus.cc:2258] T cbac84d66d5a46bbbcf2bacbae937567 P 21a48e797c1942b4bfa349ac15116147 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 23:14:31.648360 27852 ts_tablet_manager.cc:1905] T cbac84d66d5a46bbbcf2bacbae937567 P 21a48e797c1942b4bfa349ac15116147: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260923 23:14:31.648988 27200 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.227.193:38023 (error 108)
I20260923 23:14:31.649645 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 27428
W20260923 23:14:31.649662 27855 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 23:14:31.649395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.193:38023 (local address 127.0.0.1:45422)
0923 23:14:31.649528 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:31.649579 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.193:38023: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":63,"threads_started":1}
W20260923 23:14:31.651022 27200 catalog_manager.cc:4584] TS b4a935543f574f50844743bce28cbab2 (127.24.227.193:38023): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet cbac84d66d5a46bbbcf2bacbae937567: Network error: Client connection negotiation failed: client connection to 127.24.227.193:38023: connect: Connection reset by peer (error 104)
I20260923 23:14:31.655421 27855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:31.655295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.194:45731 (local address 127.0.0.1:56074)
0923 23:14:31.655339 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:31.655383 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.194:45731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260923 23:14:31.655416 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 27568
I20260923 23:14:31.659793 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 27184
2026-09-23T23:14:31Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (5794 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790205259943518-25487-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 23:14:31 dist-test-slave-754n krb5kdc[27858](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 23:14:31 dist-test-slave-754n krb5kdc[27858](info): set up 2 sockets
Sep 23 23:14:31 dist-test-slave-754n krb5kdc[27858](info): commencing operation
krb5kdc: starting...
W20260923 23:14:32.659283 25487 mini_kdc.cc:121] Time spent starting KDC: real 0.979s user 0.003s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790205259943518-25487-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790205259943518-25487-0/krb5kdc/test-user.keytab.
Sep 23 23:14:32 dist-test-slave-754n krb5kdc[27858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205272, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T23:14:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T23:14:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.227.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.227.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.227.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab.
Entry for principal kudu/127.24.227.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab.
I20260923 23:14:32.771323 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790205259943518-25487-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790205259943518-25487-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790205259943518-25487-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.254:36919
--webserver_interface=127.24.227.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab
--principal=kudu/127.24.227.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:43543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.227.254:36919
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:14:32.850380 27871 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:14:32.850560 27871 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:14:32.850581 27871 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:14:32.851636 27871 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 23:14:32.851671 27871 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:14:32.851686 27871 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:14:32.851698 27871 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 23:14:32.851711 27871 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 23:14:32.853149 27871 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:43543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790205259943518-25487-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790205259943518-25487-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.227.254:36919
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab
--principal=kudu/127.24.227.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.254:36919
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.227.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27871
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790205259943518-25487-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:14:32.853324 27871 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:14:32.853470 27871 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:14:32.857916 27877 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:14:32.858024 27871 server_base.cc:1034] running on GCE node
W20260923 23:14:32.857926 27879 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:32.857964 27876 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:14:32.858438 27871 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:14:32.858737 27871 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:14:32.859939 27871 hybrid_clock.cc:648] HybridClock initialized: now 1790205272859914 us; error 45 us; skew 500 ppm
Sep 23 23:14:32 dist-test-slave-754n krb5kdc[27858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205272, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:14:32.862423 27871 init.cc:377] Logged in from keytab as kudu/127.24.227.254@KRBTEST.COM (short username kudu)
I20260923 23:14:32.862862 27871 webserver.cc:460] Webserver started at http://127.24.227.254:42679/ using document root <none> and password file <none>
I20260923 23:14:32.863076 27871 fs_manager.cc:359] Metadata directory not provided
I20260923 23:14:32.863142 27871 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:14:32.863279 27871 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:14:32.864317 27871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790205259943518-25487-0/minicluster-data/master-0/data/instance:
uuid: "d972f0d1b5fe43d28f0398cd2cb903d6"
format_stamp: "Formatted at 2026-09-23 23:14:32 on dist-test-slave-754n"
server_key: "1a825ccfeeb0255d776b3783b186262e"
server_key_iv: "7bfff856616e7e4c70950495662ac4cc"
server_key_version: "encryptionkey@0"
I20260923 23:14:32.864707 27871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790205259943518-25487-0/minicluster-data/master-0/wal/instance:
uuid: "d972f0d1b5fe43d28f0398cd2cb903d6"
format_stamp: "Formatted at 2026-09-23 23:14:32 on dist-test-slave-754n"
server_key: "1a825ccfeeb0255d776b3783b186262e"
server_key_iv: "7bfff856616e7e4c70950495662ac4cc"
server_key_version: "encryptionkey@0"
I20260923 23:14:32.865940 27871 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 23:14:32.866634 27886 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:32.866791 27871 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 23:14:32.866885 27871 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790205259943518-25487-0/minicluster-data/master-0/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790205259943518-25487-0/minicluster-data/master-0/wal
uuid: "d972f0d1b5fe43d28f0398cd2cb903d6"
format_stamp: "Formatted at 2026-09-23 23:14:32 on dist-test-slave-754n"
server_key: "1a825ccfeeb0255d776b3783b186262e"
server_key_iv: "7bfff856616e7e4c70950495662ac4cc"
server_key_version: "encryptionkey@0"
I20260923 23:14:32.866966 27871 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790205259943518-25487-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790205259943518-25487-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790205259943518-25487-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:14:32.884763 27871 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:14:32.886839 27871 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:14:32.886992 27871 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:14:32.891045 27871 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.254:36919
I20260923 23:14:32.891093 27938 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.254:36919 every 8 connection(s)
I20260923 23:14:32.891422 27871 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
I20260923 23:14:32.892294 27939 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:14:32.894356 27939 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d972f0d1b5fe43d28f0398cd2cb903d6: Bootstrap starting.
I20260923 23:14:32.894830 27939 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d972f0d1b5fe43d28f0398cd2cb903d6: Neither blocks nor log segments found. Creating new log.
I20260923 23:14:32.895061 27939 log.cc:824] T 00000000000000000000000000000000 P d972f0d1b5fe43d28f0398cd2cb903d6: Log is configured to *not* fsync() on all Append() calls
I20260923 23:14:32.895536 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 27871
I20260923 23:14:32.895632 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790205259943518-25487-0/minicluster-data/master-0/wal/instance
I20260923 23:14:32.895792 27939 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d972f0d1b5fe43d28f0398cd2cb903d6: No bootstrap required, opened a new log
I20260923 23:14:32.895800 25487 external_mini_cluster.cc:1428] Setting key 30a876e5c49a0f775d411da99bac0c04
I20260923 23:14:32.898278 27939 raft_consensus.cc:348] T 00000000000000000000000000000000 P d972f0d1b5fe43d28f0398cd2cb903d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d972f0d1b5fe43d28f0398cd2cb903d6" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 36919 } }
I20260923 23:14:32.898402 27939 raft_consensus.cc:374] T 00000000000000000000000000000000 P d972f0d1b5fe43d28f0398cd2cb903d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:14:32.898435 27939 raft_consensus.cc:729] T 00000000000000000000000000000000 P d972f0d1b5fe43d28f0398cd2cb903d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d972f0d1b5fe43d28f0398cd2cb903d6, State: Initialized, Role: FOLLOWER
I20260923 23:14:32.898550 27939 consensus_queue.cc:260] T 00000000000000000000000000000000 P d972f0d1b5fe43d28f0398cd2cb903d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d972f0d1b5fe43d28f0398cd2cb903d6" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 36919 } }
I20260923 23:14:32.898622 27939 raft_consensus.cc:388] T 00000000000000000000000000000000 P d972f0d1b5fe43d28f0398cd2cb903d6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 23:14:32.898645 27939 raft_consensus.cc:482] T 00000000000000000000000000000000 P d972f0d1b5fe43d28f0398cd2cb903d6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 23:14:32.898706 27939 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d972f0d1b5fe43d28f0398cd2cb903d6 [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:14:32.899232 27939 raft_consensus.cc:504] T 00000000000000000000000000000000 P d972f0d1b5fe43d28f0398cd2cb903d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d972f0d1b5fe43d28f0398cd2cb903d6" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 36919 } }
I20260923 23:14:32.899331 27939 leader_election.cc:302] T 00000000000000000000000000000000 P d972f0d1b5fe43d28f0398cd2cb903d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d972f0d1b5fe43d28f0398cd2cb903d6; no voters:
I20260923 23:14:32.899502 27939 leader_election.cc:288] T 00000000000000000000000000000000 P d972f0d1b5fe43d28f0398cd2cb903d6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 23:14:32.899541 27944 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d972f0d1b5fe43d28f0398cd2cb903d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 23:14:32.899739 27944 raft_consensus.cc:686] T 00000000000000000000000000000000 P d972f0d1b5fe43d28f0398cd2cb903d6 [term 1 LEADER]: Becoming Leader. State: Replica: d972f0d1b5fe43d28f0398cd2cb903d6, State: Running, Role: LEADER
I20260923 23:14:32.899824 27939 sys_catalog.cc:564] T 00000000000000000000000000000000 P d972f0d1b5fe43d28f0398cd2cb903d6 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 23:14:32.899911 27944 consensus_queue.cc:237] T 00000000000000000000000000000000 P d972f0d1b5fe43d28f0398cd2cb903d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d972f0d1b5fe43d28f0398cd2cb903d6" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 36919 } }
I20260923 23:14:32.900307 27944 sys_catalog.cc:455] T 00000000000000000000000000000000 P d972f0d1b5fe43d28f0398cd2cb903d6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d972f0d1b5fe43d28f0398cd2cb903d6. Latest consensus state: current_term: 1 leader_uuid: "d972f0d1b5fe43d28f0398cd2cb903d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d972f0d1b5fe43d28f0398cd2cb903d6" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 36919 } } }
I20260923 23:14:32.900315 27945 sys_catalog.cc:455] T 00000000000000000000000000000000 P d972f0d1b5fe43d28f0398cd2cb903d6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d972f0d1b5fe43d28f0398cd2cb903d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d972f0d1b5fe43d28f0398cd2cb903d6" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 36919 } } }
I20260923 23:14:32.900476 27945 sys_catalog.cc:458] T 00000000000000000000000000000000 P d972f0d1b5fe43d28f0398cd2cb903d6 [sys.catalog]: This master's current role is: LEADER
I20260923 23:14:32.900672 27944 sys_catalog.cc:458] T 00000000000000000000000000000000 P d972f0d1b5fe43d28f0398cd2cb903d6 [sys.catalog]: This master's current role is: LEADER
I20260923 23:14:32.900807 27953 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 23:14:32.901276 27953 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 23:14:32.903142 27953 catalog_manager.cc:1347] Generated new cluster ID: 2cb73408d7d64797a89a42bb442728ef
I20260923 23:14:32.903244 27953 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 23:14:32 dist-test-slave-754n krb5kdc[27858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205272, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
I20260923 23:14:32.907320 27943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:32.896306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54848 (local address 127.24.227.254:36919)
0923 23:14:32.896465 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:32.896472 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:32.896498 (+ 26us) server_negotiation.cc:407] Connection header received
0923 23:14:32.897171 (+ 673us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:32.897189 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:32.897236 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:32.897346 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:32.898155 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:32.899473 (+ 1318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:32.900636 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:32.901242 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:32.901355 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:32.904054 (+ 2699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:32.904058 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:32.904058 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:32.904077 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:32.905738 (+ 1661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:32.906152 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:32.906155 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:32.906155 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:32.906238 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:32.906447 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:32.906448 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:32.906448 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:32.906654 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:32.906817 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:32.906914 (+ 97us) server_negotiation.cc:299] Negotiation successful
0923 23:14:32.907054 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":82,"threads_started":1}
I20260923 23:14:32.914032 27953 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 23:14:32.914495 27953 catalog_manager.cc:1504] Loading token signing keys...
I20260923 23:14:32.922226 27953 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d972f0d1b5fe43d28f0398cd2cb903d6: Generated new TSK 0
I20260923 23:14:32.922389 27953 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.227.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.227.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.227.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.193.keytab.
Entry for principal kudu/127.24.227.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.193.keytab.
I20260923 23:14:32.948441 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790205259943518-25487-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790205259943518-25487-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790205259943518-25487-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790205259943518-25487-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.227.193:0
--local_ip_for_outbound_sockets=127.24.227.193
--webserver_interface=127.24.227.193
--webserver_port=0
--tserver_master_addrs=127.24.227.254:36919
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.193.keytab
--principal=kudu/127.24.227.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:43543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:14:33.024149 27965 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:14:33.024333 27965 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:14:33.024354 27965 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:14:33.025502 27965 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:14:33.025542 27965 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:14:33.025568 27965 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.227.193
I20260923 23:14:33.026983 27965 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:43543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790205259943518-25487-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790205259943518-25487-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.193.keytab
--principal=kudu/127.24.227.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.227.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790205259943518-25487-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.227.193
--webserver_port=0
--tserver_master_addrs=127.24.227.254:36919
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27965
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.227.193
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790205259943518-25487-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:14:33.027220 27965 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:14:33.027443 27965 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:14:33.032145 27970 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:14:33.032229 27965 server_base.cc:1034] running on GCE node
W20260923 23:14:33.032150 27973 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:33.032330 27971 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:14:33.032574 27965 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:14:33.032763 27965 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:14:33.033896 27965 hybrid_clock.cc:648] HybridClock initialized: now 1790205273033886 us; error 24 us; skew 500 ppm
Sep 23 23:14:33 dist-test-slave-754n krb5kdc[27858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205273, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:14:33.036232 27965 init.cc:377] Logged in from keytab as kudu/127.24.227.193@KRBTEST.COM (short username kudu)
I20260923 23:14:33.036643 27965 webserver.cc:460] Webserver started at http://127.24.227.193:38457/ using document root <none> and password file <none>
I20260923 23:14:33.036872 27965 fs_manager.cc:359] Metadata directory not provided
I20260923 23:14:33.036954 27965 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:14:33.037104 27965 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:14:33.038103 27965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790205259943518-25487-0/minicluster-data/ts-0/data/instance:
uuid: "17bad9dd176b4cfdb92cab35f0d608eb"
format_stamp: "Formatted at 2026-09-23 23:14:33 on dist-test-slave-754n"
server_key: "79352b87ce70c37a970b7ab530c6a47e"
server_key_iv: "00f5ab36af1dc5d510a30346e6dd7f3e"
server_key_version: "encryptionkey@0"
I20260923 23:14:33.038452 27965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790205259943518-25487-0/minicluster-data/ts-0/wal/instance:
uuid: "17bad9dd176b4cfdb92cab35f0d608eb"
format_stamp: "Formatted at 2026-09-23 23:14:33 on dist-test-slave-754n"
server_key: "79352b87ce70c37a970b7ab530c6a47e"
server_key_iv: "00f5ab36af1dc5d510a30346e6dd7f3e"
server_key_version: "encryptionkey@0"
I20260923 23:14:33.039721 27965 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 23:14:33.040485 27980 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:33.040638 27965 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 23:14:33.040751 27965 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790205259943518-25487-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790205259943518-25487-0/minicluster-data/ts-0/wal
uuid: "17bad9dd176b4cfdb92cab35f0d608eb"
format_stamp: "Formatted at 2026-09-23 23:14:33 on dist-test-slave-754n"
server_key: "79352b87ce70c37a970b7ab530c6a47e"
server_key_iv: "00f5ab36af1dc5d510a30346e6dd7f3e"
server_key_version: "encryptionkey@0"
I20260923 23:14:33.040835 27965 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790205259943518-25487-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790205259943518-25487-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790205259943518-25487-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:14:33.049538 27965 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:14:33.051182 27965 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:14:33.051331 27965 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:14:33.051522 27965 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 23:14:33.051815 27965 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 23:14:33.051867 27965 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:33.051920 27965 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 23:14:33.051952 27965 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:33.057437 27965 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.193:35311
I20260923 23:14:33.057534 28093 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.193:35311 every 8 connection(s)
I20260923 23:14:33.057803 27965 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790205259943518-25487-0/minicluster-data/ts-0/data/info.pb
I20260923 23:14:33.062505 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 27965
I20260923 23:14:33.062609 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790205259943518-25487-0/minicluster-data/ts-0/wal/instance
I20260923 23:14:33.062781 25487 external_mini_cluster.cc:1428] Setting key 531f01ade45ae950bd21509f1aec8e54
Sep 23 23:14:33 dist-test-slave-754n krb5kdc[27858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205273, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.193@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
I20260923 23:14:33.067034 27943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:33.058532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.193:58439 (local address 127.24.227.254:36919)
0923 23:14:33.058569 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:33.058571 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:33.058835 (+ 264us) server_negotiation.cc:407] Connection header received
0923 23:14:33.059266 (+ 431us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:33.059267 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:33.059277 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:33.059333 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:33.060286 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:33.060769 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:33.061676 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:33.061856 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:33.061904 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:33.064191 (+ 2287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:33.064193 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:33.064193 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:33.064201 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:33.065714 (+ 1513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:33.066111 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:33.066112 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:33.066113 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:33.066174 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:33.066402 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:33.066403 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:33.066403 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:33.066544 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:33.066611 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:33.066827 (+ 216us) server_negotiation.cc:299] Negotiation successful
0923 23:14:33.066919 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260923 23:14:33.067071 28096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:33.058505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:36919 (local address 127.24.227.193:58439)
0923 23:14:33.058717 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:33.058755 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:33.059188 (+ 433us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:33.059378 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:33.059381 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:33.060001 (+ 620us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:14:33.060200 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:33.060206 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:33.060849 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:33.060850 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:33.061602 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:33.061603 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:33.061923 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:33.061924 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:33.062020 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:33.062493 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:14:33.062501 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:14:33.064099 (+ 1598us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:14:33.065785 (+ 1686us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:33.065790 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:33.065791 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:33.066033 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:33.066246 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:33.066246 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:33.066247 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:33.066340 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:33.066605 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:14:33.066609 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:14:33.066704 (+ 95us) client_negotiation.cc:769] Sending connection context
0923 23:14:33.066780 (+ 76us) client_negotiation.cc:240] Negotiation successful
0923 23:14:33.066883 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":138,"threads_started":1}
I20260923 23:14:33.067582 28094 heartbeater.cc:347] Connected to a master server at 127.24.227.254:36919
I20260923 23:14:33.067665 28094 heartbeater.cc:464] Registering TS with master...
I20260923 23:14:33.067826 28094 heartbeater.cc:511] Master 127.24.227.254:36919 requested a full tablet report, sending...
I20260923 23:14:33.068343 27903 ts_manager.cc:194] Registered new tserver with Master: 17bad9dd176b4cfdb92cab35f0d608eb (127.24.227.193:35311)
I20260923 23:14:33.068945 27903 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.227.193@KRBTEST.COM'} at 127.24.227.193:58439
WARNING: no policy specified for kudu/127.24.227.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.227.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.227.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.194.keytab.
Entry for principal kudu/127.24.227.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.194.keytab.
I20260923 23:14:33.087855 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790205259943518-25487-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790205259943518-25487-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790205259943518-25487-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790205259943518-25487-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.227.194:0
--local_ip_for_outbound_sockets=127.24.227.194
--webserver_interface=127.24.227.194
--webserver_port=0
--tserver_master_addrs=127.24.227.254:36919
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.194.keytab
--principal=kudu/127.24.227.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:43543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:14:33.163853 28099 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:14:33.164088 28099 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:14:33.164122 28099 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:14:33.165390 28099 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:14:33.165452 28099 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:14:33.165494 28099 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.227.194
I20260923 23:14:33.166887 28099 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:43543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790205259943518-25487-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790205259943518-25487-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.194.keytab
--principal=kudu/127.24.227.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.227.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790205259943518-25487-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.227.194
--webserver_port=0
--tserver_master_addrs=127.24.227.254:36919
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28099
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.227.194
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790205259943518-25487-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:14:33.167122 28099 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:14:33.167353 28099 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:14:33.171674 28104 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:33.171681 28107 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:33.171833 28105 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:14:33.172066 28099 server_base.cc:1034] running on GCE node
I20260923 23:14:33.172282 28099 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:14:33.172511 28099 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:14:33.173650 28099 hybrid_clock.cc:648] HybridClock initialized: now 1790205273173642 us; error 34 us; skew 500 ppm
Sep 23 23:14:33 dist-test-slave-754n krb5kdc[27858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205273, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:14:33.175951 28099 init.cc:377] Logged in from keytab as kudu/127.24.227.194@KRBTEST.COM (short username kudu)
I20260923 23:14:33.176394 28099 webserver.cc:460] Webserver started at http://127.24.227.194:34467/ using document root <none> and password file <none>
I20260923 23:14:33.176622 28099 fs_manager.cc:359] Metadata directory not provided
I20260923 23:14:33.176687 28099 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:14:33.176815 28099 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:14:33.177750 28099 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790205259943518-25487-0/minicluster-data/ts-1/data/instance:
uuid: "0e5dfedf4aee43968e551c4f02b4a1b8"
format_stamp: "Formatted at 2026-09-23 23:14:33 on dist-test-slave-754n"
server_key: "1d890570c929dda8b31dbf31ab7f6714"
server_key_iv: "00b5566ed2337f2b6645f75a2b36e1b8"
server_key_version: "encryptionkey@0"
I20260923 23:14:33.178092 28099 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790205259943518-25487-0/minicluster-data/ts-1/wal/instance:
uuid: "0e5dfedf4aee43968e551c4f02b4a1b8"
format_stamp: "Formatted at 2026-09-23 23:14:33 on dist-test-slave-754n"
server_key: "1d890570c929dda8b31dbf31ab7f6714"
server_key_iv: "00b5566ed2337f2b6645f75a2b36e1b8"
server_key_version: "encryptionkey@0"
I20260923 23:14:33.179330 28099 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 23:14:33.180083 28114 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:33.180292 28099 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 23:14:33.180387 28099 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790205259943518-25487-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790205259943518-25487-0/minicluster-data/ts-1/wal
uuid: "0e5dfedf4aee43968e551c4f02b4a1b8"
format_stamp: "Formatted at 2026-09-23 23:14:33 on dist-test-slave-754n"
server_key: "1d890570c929dda8b31dbf31ab7f6714"
server_key_iv: "00b5566ed2337f2b6645f75a2b36e1b8"
server_key_version: "encryptionkey@0"
I20260923 23:14:33.180471 28099 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790205259943518-25487-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790205259943518-25487-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790205259943518-25487-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:14:33.192987 28099 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:14:33.194591 28099 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:14:33.194742 28099 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:14:33.194945 28099 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 23:14:33.195217 28099 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 23:14:33.195277 28099 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:33.195309 28099 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 23:14:33.195334 28099 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:33.201249 28099 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.194:36911
I20260923 23:14:33.201318 28227 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.194:36911 every 8 connection(s)
I20260923 23:14:33.201660 28099 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790205259943518-25487-0/minicluster-data/ts-1/data/info.pb
Sep 23 23:14:33 dist-test-slave-754n krb5kdc[27858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205273, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.194@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
I20260923 23:14:33.209955 27943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:33.202362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.194:46655 (local address 127.24.227.254:36919)
0923 23:14:33.202441 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:33.202443 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:33.202635 (+ 192us) server_negotiation.cc:407] Connection header received
0923 23:14:33.203063 (+ 428us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:33.203064 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:33.203073 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:33.203124 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:33.204041 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:33.204522 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:33.205440 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:33.205628 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:33.205677 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:33.207708 (+ 2031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:33.207709 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:33.207709 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:33.207714 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:33.208912 (+ 1198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:33.209249 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:33.209249 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:33.209249 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:33.209295 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:33.209490 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:33.209490 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:33.209491 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:33.209581 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:33.209628 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:33.209844 (+ 216us) server_negotiation.cc:299] Negotiation successful
0923 23:14:33.209896 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260923 23:14:33.210096 28230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:33.202340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:36919 (local address 127.24.227.194:46655)
0923 23:14:33.202515 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:33.202546 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:33.202985 (+ 439us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:33.203171 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:33.203174 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:33.203695 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:14:33.203955 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:33.203962 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:33.204594 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:33.204595 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:33.205369 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:33.205371 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:33.205697 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:33.205697 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:33.205797 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:33.206307 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:14:33.206315 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:14:33.207632 (+ 1317us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:14:33.208977 (+ 1345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:33.208982 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:33.208984 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:33.209182 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:33.209350 (+ 168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:33.209351 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:33.209351 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:33.209439 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:33.209632 (+ 193us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:14:33.209635 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:14:33.209738 (+ 103us) client_negotiation.cc:769] Sending connection context
0923 23:14:33.209813 (+ 75us) client_negotiation.cc:240] Negotiation successful
0923 23:14:33.209918 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":103,"threads_started":1}
I20260923 23:14:33.210553 28228 heartbeater.cc:347] Connected to a master server at 127.24.227.254:36919
I20260923 23:14:33.210635 28228 heartbeater.cc:464] Registering TS with master...
I20260923 23:14:33.210772 28228 heartbeater.cc:511] Master 127.24.227.254:36919 requested a full tablet report, sending...
I20260923 23:14:33.211141 27903 ts_manager.cc:194] Registered new tserver with Master: 0e5dfedf4aee43968e551c4f02b4a1b8 (127.24.227.194:36911)
I20260923 23:14:33.211493 27903 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.227.194@KRBTEST.COM'} at 127.24.227.194:46655
I20260923 23:14:33.211781 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 28099
I20260923 23:14:33.211903 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790205259943518-25487-0/minicluster-data/ts-1/wal/instance
I20260923 23:14:33.212097 25487 external_mini_cluster.cc:1428] Setting key 37a32f5ae303f7829937951b81554d3e
WARNING: no policy specified for kudu/127.24.227.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.227.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.227.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.195.keytab.
Entry for principal kudu/127.24.227.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.195.keytab.
I20260923 23:14:33.234278 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790205259943518-25487-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790205259943518-25487-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790205259943518-25487-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790205259943518-25487-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.227.195:0
--local_ip_for_outbound_sockets=127.24.227.195
--webserver_interface=127.24.227.195
--webserver_port=0
--tserver_master_addrs=127.24.227.254:36919
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.195.keytab
--principal=kudu/127.24.227.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:43543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:14:33.309150 28233 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:14:33.309329 28233 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:14:33.309351 28233 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:14:33.310492 28233 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:14:33.310524 28233 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:14:33.310549 28233 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.227.195
I20260923 23:14:33.311848 28233 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:43543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790205259943518-25487-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790205259943518-25487-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.195.keytab
--principal=kudu/127.24.227.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.227.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790205259943518-25487-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.227.195
--webserver_port=0
--tserver_master_addrs=127.24.227.254:36919
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28233
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.227.195
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790205259943518-25487-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:14:33.312021 28233 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:14:33.312196 28233 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:14:33.316660 28238 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:33.316661 28239 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:33.316681 28241 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:14:33.316985 28233 server_base.cc:1034] running on GCE node
I20260923 23:14:33.317164 28233 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:14:33.317325 28233 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:14:33.318455 28233 hybrid_clock.cc:648] HybridClock initialized: now 1790205273318444 us; error 27 us; skew 500 ppm
Sep 23 23:14:33 dist-test-slave-754n krb5kdc[27858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205273, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:14:33.320729 28233 init.cc:377] Logged in from keytab as kudu/127.24.227.195@KRBTEST.COM (short username kudu)
I20260923 23:14:33.321164 28233 webserver.cc:460] Webserver started at http://127.24.227.195:42151/ using document root <none> and password file <none>
I20260923 23:14:33.321372 28233 fs_manager.cc:359] Metadata directory not provided
I20260923 23:14:33.321413 28233 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:14:33.321564 28233 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:14:33.322518 28233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790205259943518-25487-0/minicluster-data/ts-2/data/instance:
uuid: "ccebe956c7d743e3bdf7b7c902cf7bf9"
format_stamp: "Formatted at 2026-09-23 23:14:33 on dist-test-slave-754n"
server_key: "98470d6298fa9fdd5bb41486bfab3127"
server_key_iv: "9fb253363e502b5e0738f1c46b02566e"
server_key_version: "encryptionkey@0"
I20260923 23:14:33.322853 28233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790205259943518-25487-0/minicluster-data/ts-2/wal/instance:
uuid: "ccebe956c7d743e3bdf7b7c902cf7bf9"
format_stamp: "Formatted at 2026-09-23 23:14:33 on dist-test-slave-754n"
server_key: "98470d6298fa9fdd5bb41486bfab3127"
server_key_iv: "9fb253363e502b5e0738f1c46b02566e"
server_key_version: "encryptionkey@0"
I20260923 23:14:33.324157 28233 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 23:14:33.324788 28248 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:33.324930 28233 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 23:14:33.325016 28233 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790205259943518-25487-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790205259943518-25487-0/minicluster-data/ts-2/wal
uuid: "ccebe956c7d743e3bdf7b7c902cf7bf9"
format_stamp: "Formatted at 2026-09-23 23:14:33 on dist-test-slave-754n"
server_key: "98470d6298fa9fdd5bb41486bfab3127"
server_key_iv: "9fb253363e502b5e0738f1c46b02566e"
server_key_version: "encryptionkey@0"
I20260923 23:14:33.325089 28233 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790205259943518-25487-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790205259943518-25487-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790205259943518-25487-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:14:33.340129 28233 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:14:33.341781 28233 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:14:33.341930 28233 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:14:33.342141 28233 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 23:14:33.342406 28233 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 23:14:33.342458 28233 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:33.342495 28233 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 23:14:33.342521 28233 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:33.348071 28233 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.195:40541
I20260923 23:14:33.348110 28361 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.195:40541 every 8 connection(s)
I20260923 23:14:33.348380 28233 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790205259943518-25487-0/minicluster-data/ts-2/data/info.pb
Sep 23 23:14:33 dist-test-slave-754n krb5kdc[27858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205273, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.195@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
I20260923 23:14:33.356683 27943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:33.348987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.195:44499 (local address 127.24.227.254:36919)
0923 23:14:33.349026 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:33.349027 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:33.349201 (+ 174us) server_negotiation.cc:407] Connection header received
0923 23:14:33.349682 (+ 481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:33.349684 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:33.349696 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:33.349745 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:33.350586 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:33.351107 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:33.352171 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:33.352375 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:33.352459 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:33.354323 (+ 1864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:33.354325 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:33.354325 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:33.354332 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:33.355635 (+ 1303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:33.356022 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:33.356023 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:33.356023 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:33.356073 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:33.356260 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:33.356260 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:33.356261 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:33.356346 (+ 85us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:33.356389 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:33.356572 (+ 183us) server_negotiation.cc:299] Negotiation successful
0923 23:14:33.356625 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260923 23:14:33.356896 28364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:33.348961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:36919 (local address 127.24.227.195:44499)
0923 23:14:33.349088 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:33.349125 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:33.349575 (+ 450us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:33.349805 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:33.349809 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:33.350321 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:14:33.350489 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:33.350493 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:33.351202 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:33.351203 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:33.352062 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:33.352064 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:33.352466 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:33.352468 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:33.352594 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:33.352961 (+ 367us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:14:33.352968 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:14:33.354240 (+ 1272us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:14:33.355706 (+ 1466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:33.355711 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:33.355712 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:33.355971 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:33.356131 (+ 160us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:33.356132 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:33.356132 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:33.356219 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:33.356392 (+ 173us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:14:33.356396 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:14:33.356490 (+ 94us) client_negotiation.cc:769] Sending connection context
0923 23:14:33.356563 (+ 73us) client_negotiation.cc:240] Negotiation successful
0923 23:14:33.356663 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":64,"threads_started":1}
I20260923 23:14:33.357429 28362 heartbeater.cc:347] Connected to a master server at 127.24.227.254:36919
I20260923 23:14:33.357522 28362 heartbeater.cc:464] Registering TS with master...
I20260923 23:14:33.357695 28362 heartbeater.cc:511] Master 127.24.227.254:36919 requested a full tablet report, sending...
I20260923 23:14:33.358033 27903 ts_manager.cc:194] Registered new tserver with Master: ccebe956c7d743e3bdf7b7c902cf7bf9 (127.24.227.195:40541)
I20260923 23:14:33.358282 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 28233
I20260923 23:14:33.358381 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790205259943518-25487-0/minicluster-data/ts-2/wal/instance
I20260923 23:14:33.358566 25487 external_mini_cluster.cc:1428] Setting key b26d2748b2d0b5f7719e3eac95811b0d
I20260923 23:14:33.358572 27903 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.227.195@KRBTEST.COM'} at 127.24.227.195:44499
I20260923 23:14:33.359304 25487 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260923 23:14:33.369624 27943 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 23:14:33.368949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54854 (local address 127.24.227.254:36919)
0923 23:14:33.369005 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:33.369007 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:33.369102 (+ 95us) server_negotiation.cc:407] Connection header received
0923 23:14:33.369147 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:33.369148 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:33.369154 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:33.369199 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:33.369587 (+ 388us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54854: BlockingRecv error: recv got EOF from 127.0.0.1:54854 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
I20260923 23:14:33.370731 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 27965
I20260923 23:14:33.375095 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 28099
I20260923 23:14:33.379151 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 28233
I20260923 23:14:33.383399 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 27871
2026-09-23T23:14:33Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (1722 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790205259943518-25487-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 23:14:33 dist-test-slave-754n krb5kdc[28374](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 23:14:33 dist-test-slave-754n krb5kdc[28374](info): set up 2 sockets
Sep 23 23:14:33 dist-test-slave-754n krb5kdc[28374](info): commencing operation
krb5kdc: starting...
W20260923 23:14:34.391791 25487 mini_kdc.cc:121] Time spent starting KDC: real 0.989s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790205259943518-25487-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790205259943518-25487-0/krb5kdc/test-user.keytab.
Sep 23 23:14:34 dist-test-slave-754n krb5kdc[28374](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205274, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T23:14:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T23:14:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.227.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.227.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.227.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab.
Entry for principal kudu/127.24.227.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab.
I20260923 23:14:34.496346 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790205259943518-25487-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790205259943518-25487-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790205259943518-25487-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.254:35105
--webserver_interface=127.24.227.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab
--principal=kudu/127.24.227.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:37043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.227.254:35105
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:14:34.570585 28387 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:14:34.570817 28387 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:14:34.570854 28387 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:14:34.572078 28387 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 23:14:34.572134 28387 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:14:34.572158 28387 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:14:34.572187 28387 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 23:14:34.572216 28387 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 23:14:34.573568 28387 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:37043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790205259943518-25487-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790205259943518-25487-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.227.254:35105
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab
--principal=kudu/127.24.227.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.254:35105
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.227.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28387
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790205259943518-25487-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:14:34.573817 28387 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:14:34.574023 28387 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:14:34.578233 28392 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:34.578250 28395 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:34.578248 28393 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:14:34.578547 28387 server_base.cc:1034] running on GCE node
I20260923 23:14:34.578776 28387 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:14:34.579038 28387 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:14:34.580193 28387 hybrid_clock.cc:648] HybridClock initialized: now 1790205274580176 us; error 29 us; skew 500 ppm
Sep 23 23:14:34 dist-test-slave-754n krb5kdc[28374](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205274, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:14:34.582470 28387 init.cc:377] Logged in from keytab as kudu/127.24.227.254@KRBTEST.COM (short username kudu)
I20260923 23:14:34.582890 28387 webserver.cc:460] Webserver started at http://127.24.227.254:36515/ using document root <none> and password file <none>
I20260923 23:14:34.583112 28387 fs_manager.cc:359] Metadata directory not provided
I20260923 23:14:34.583175 28387 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:14:34.583329 28387 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:14:34.584321 28387 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790205259943518-25487-0/minicluster-data/master-0/data/instance:
uuid: "f547a4df4e9840f88020987c9e7333ad"
format_stamp: "Formatted at 2026-09-23 23:14:34 on dist-test-slave-754n"
server_key: "7b5095c011453ea18f76ef00c9b4cc13"
server_key_iv: "cc414cecea00cb737445192510d81e01"
server_key_version: "encryptionkey@0"
I20260923 23:14:34.584656 28387 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790205259943518-25487-0/minicluster-data/master-0/wal/instance:
uuid: "f547a4df4e9840f88020987c9e7333ad"
format_stamp: "Formatted at 2026-09-23 23:14:34 on dist-test-slave-754n"
server_key: "7b5095c011453ea18f76ef00c9b4cc13"
server_key_iv: "cc414cecea00cb737445192510d81e01"
server_key_version: "encryptionkey@0"
I20260923 23:14:34.585906 28387 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 23:14:34.586685 28402 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:34.586858 28387 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 23:14:34.586968 28387 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790205259943518-25487-0/minicluster-data/master-0/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790205259943518-25487-0/minicluster-data/master-0/wal
uuid: "f547a4df4e9840f88020987c9e7333ad"
format_stamp: "Formatted at 2026-09-23 23:14:34 on dist-test-slave-754n"
server_key: "7b5095c011453ea18f76ef00c9b4cc13"
server_key_iv: "cc414cecea00cb737445192510d81e01"
server_key_version: "encryptionkey@0"
I20260923 23:14:34.587039 28387 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790205259943518-25487-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790205259943518-25487-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790205259943518-25487-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:14:34.596041 28387 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:14:34.597748 28387 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:14:34.597951 28387 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:14:34.601871 28387 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.254:35105
I20260923 23:14:34.601912 28454 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.254:35105 every 8 connection(s)
I20260923 23:14:34.602348 28387 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
I20260923 23:14:34.602924 28455 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:14:34.604897 28455 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f547a4df4e9840f88020987c9e7333ad: Bootstrap starting.
I20260923 23:14:34.605374 28455 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f547a4df4e9840f88020987c9e7333ad: Neither blocks nor log segments found. Creating new log.
I20260923 23:14:34.605605 28455 log.cc:824] T 00000000000000000000000000000000 P f547a4df4e9840f88020987c9e7333ad: Log is configured to *not* fsync() on all Append() calls
I20260923 23:14:34.606171 28455 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f547a4df4e9840f88020987c9e7333ad: No bootstrap required, opened a new log
I20260923 23:14:34.607580 28455 raft_consensus.cc:348] T 00000000000000000000000000000000 P f547a4df4e9840f88020987c9e7333ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f547a4df4e9840f88020987c9e7333ad" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 35105 } }
I20260923 23:14:34.607707 28455 raft_consensus.cc:374] T 00000000000000000000000000000000 P f547a4df4e9840f88020987c9e7333ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:14:34.607726 28455 raft_consensus.cc:729] T 00000000000000000000000000000000 P f547a4df4e9840f88020987c9e7333ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f547a4df4e9840f88020987c9e7333ad, State: Initialized, Role: FOLLOWER
I20260923 23:14:34.607846 28455 consensus_queue.cc:260] T 00000000000000000000000000000000 P f547a4df4e9840f88020987c9e7333ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f547a4df4e9840f88020987c9e7333ad" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 35105 } }
I20260923 23:14:34.607963 28455 raft_consensus.cc:388] T 00000000000000000000000000000000 P f547a4df4e9840f88020987c9e7333ad [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 23:14:34.607996 28455 raft_consensus.cc:482] T 00000000000000000000000000000000 P f547a4df4e9840f88020987c9e7333ad [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 23:14:34.608026 28455 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f547a4df4e9840f88020987c9e7333ad [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:14:34.608738 28455 raft_consensus.cc:504] T 00000000000000000000000000000000 P f547a4df4e9840f88020987c9e7333ad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f547a4df4e9840f88020987c9e7333ad" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 35105 } }
I20260923 23:14:34.608814 28455 leader_election.cc:302] T 00000000000000000000000000000000 P f547a4df4e9840f88020987c9e7333ad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f547a4df4e9840f88020987c9e7333ad; no voters:
I20260923 23:14:34.608916 28455 leader_election.cc:288] T 00000000000000000000000000000000 P f547a4df4e9840f88020987c9e7333ad [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 23:14:34.608985 28458 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f547a4df4e9840f88020987c9e7333ad [term 1 FOLLOWER]: Leader election won for term 1
I20260923 23:14:34.609131 28458 raft_consensus.cc:686] T 00000000000000000000000000000000 P f547a4df4e9840f88020987c9e7333ad [term 1 LEADER]: Becoming Leader. State: Replica: f547a4df4e9840f88020987c9e7333ad, State: Running, Role: LEADER
I20260923 23:14:34.609143 28455 sys_catalog.cc:564] T 00000000000000000000000000000000 P f547a4df4e9840f88020987c9e7333ad [sys.catalog]: configured and running, proceeding with master startup.
I20260923 23:14:34.609221 28458 consensus_queue.cc:237] T 00000000000000000000000000000000 P f547a4df4e9840f88020987c9e7333ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f547a4df4e9840f88020987c9e7333ad" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 35105 } }
I20260923 23:14:34.609572 28459 sys_catalog.cc:455] T 00000000000000000000000000000000 P f547a4df4e9840f88020987c9e7333ad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f547a4df4e9840f88020987c9e7333ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f547a4df4e9840f88020987c9e7333ad" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 35105 } } }
I20260923 23:14:34.609627 28460 sys_catalog.cc:455] T 00000000000000000000000000000000 P f547a4df4e9840f88020987c9e7333ad [sys.catalog]: SysCatalogTable state changed. Reason: New leader f547a4df4e9840f88020987c9e7333ad. Latest consensus state: current_term: 1 leader_uuid: "f547a4df4e9840f88020987c9e7333ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f547a4df4e9840f88020987c9e7333ad" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 35105 } } }
I20260923 23:14:34.609874 28460 sys_catalog.cc:458] T 00000000000000000000000000000000 P f547a4df4e9840f88020987c9e7333ad [sys.catalog]: This master's current role is: LEADER
I20260923 23:14:34.609855 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 28387
I20260923 23:14:34.609972 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790205259943518-25487-0/minicluster-data/master-0/wal/instance
I20260923 23:14:34.610153 25487 external_mini_cluster.cc:1428] Setting key 517abfea3b6f148ba55cc52ae39ee639
I20260923 23:14:34.609654 28459 sys_catalog.cc:458] T 00000000000000000000000000000000 P f547a4df4e9840f88020987c9e7333ad [sys.catalog]: This master's current role is: LEADER
I20260923 23:14:34.610343 28467 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 23:14:34.611217 28467 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 23:14:34.612830 28467 catalog_manager.cc:1347] Generated new cluster ID: e052b4d047554b789367f6b8b4df880a
I20260923 23:14:34.612881 28467 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 23:14:34 dist-test-slave-754n krb5kdc[28374](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205274, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
I20260923 23:14:34.619916 28476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:34.610662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52770 (local address 127.24.227.254:35105)
0923 23:14:34.610795 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:34.610814 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:34.610844 (+ 30us) server_negotiation.cc:407] Connection header received
0923 23:14:34.610973 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:34.610978 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:34.611016 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:34.611166 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:34.611945 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:34.613164 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:34.613936 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:34.614341 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:34.614450 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:34.616696 (+ 2246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:34.616700 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:34.616700 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:34.616711 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:34.618405 (+ 1694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:34.618776 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:34.618779 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:34.618779 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:34.618864 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:34.619079 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:34.619079 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:34.619080 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:34.619310 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:34.619478 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:34.619546 (+ 68us) server_negotiation.cc:299] Negotiation successful
0923 23:14:34.619678 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90,"thread_start_us":67,"threads_started":1}
I20260923 23:14:34.620820 25487 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 23:14:34.621243 28467 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 23:14:34.621905 28467 catalog_manager.cc:1504] Loading token signing keys...
I20260923 23:14:34.632493 28467 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f547a4df4e9840f88020987c9e7333ad: Generated new TSK 0
I20260923 23:14:34.632647 28467 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 23:14:34.633594 28476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:34.630202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52774 (local address 127.24.227.254:35105)
0923 23:14:34.630257 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:34.630258 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:34.630389 (+ 131us) server_negotiation.cc:407] Connection header received
0923 23:14:34.630450 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:34.630450 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:34.630460 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:34.630499 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:34.630829 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:34.631294 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:34.632116 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:34.632334 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:34.632418 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:34.633145 (+ 727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:34.633146 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:34.633146 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 23:14:34.633305 (+ 159us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:34.633317 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 23:14:34.633323 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:34.633390 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:34.633460 (+ 70us) server_negotiation.cc:299] Negotiation successful
0923 23:14:34.633481 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260923 23:14:34.635742 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 28387
2026-09-23T23:14:34Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1249 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790205259943518-25487-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 23:14:34 dist-test-slave-754n krb5kdc[28488](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 23:14:34 dist-test-slave-754n krb5kdc[28488](info): set up 2 sockets
Sep 23 23:14:34 dist-test-slave-754n krb5kdc[28488](info): commencing operation
krb5kdc: starting...
W20260923 23:14:35.639297 25487 mini_kdc.cc:121] Time spent starting KDC: real 0.986s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790205259943518-25487-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790205259943518-25487-0/krb5kdc/test-user.keytab.
Sep 23 23:14:35 dist-test-slave-754n krb5kdc[28488](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205275, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T23:14:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T23:14:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.227.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.227.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.227.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab.
Entry for principal kudu/127.24.227.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab.
I20260923 23:14:35.744601 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790205259943518-25487-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790205259943518-25487-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790205259943518-25487-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.254:38139
--webserver_interface=127.24.227.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab
--principal=kudu/127.24.227.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:46097
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.227.254:38139
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:14:35.820783 28501 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:14:35.820971 28501 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:14:35.820992 28501 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:14:35.822170 28501 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 23:14:35.822211 28501 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:14:35.822224 28501 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:14:35.822274 28501 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 23:14:35.822304 28501 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 23:14:35.823827 28501 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:46097
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790205259943518-25487-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790205259943518-25487-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.227.254:38139
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab
--principal=kudu/127.24.227.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.254:38139
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.227.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28501
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790205259943518-25487-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:14:35.824071 28501 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:14:35.824273 28501 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:14:35.828684 28506 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:35.828752 28509 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:14:35.828903 28501 server_base.cc:1034] running on GCE node
W20260923 23:14:35.828689 28507 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:14:35.829228 28501 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:14:35.829528 28501 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:14:35.830740 28501 hybrid_clock.cc:648] HybridClock initialized: now 1790205275830730 us; error 58 us; skew 500 ppm
Sep 23 23:14:35 dist-test-slave-754n krb5kdc[28488](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205275, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:14:35.838229 28501 init.cc:377] Logged in from keytab as kudu/127.24.227.254@KRBTEST.COM (short username kudu)
I20260923 23:14:35.838702 28501 webserver.cc:460] Webserver started at http://127.24.227.254:38123/ using document root <none> and password file <none>
I20260923 23:14:35.838929 28501 fs_manager.cc:359] Metadata directory not provided
I20260923 23:14:35.838992 28501 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:14:35.839138 28501 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:14:35.840171 28501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790205259943518-25487-0/minicluster-data/master-0/data/instance:
uuid: "688c347bae604d06b4fccb39d80e03bc"
format_stamp: "Formatted at 2026-09-23 23:14:35 on dist-test-slave-754n"
server_key: "1478ddc58850f1b3f39fdf40e20fb5cf"
server_key_iv: "fd171ed71d0c89dcf249ca130f389e2f"
server_key_version: "encryptionkey@0"
I20260923 23:14:35.840509 28501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790205259943518-25487-0/minicluster-data/master-0/wal/instance:
uuid: "688c347bae604d06b4fccb39d80e03bc"
format_stamp: "Formatted at 2026-09-23 23:14:35 on dist-test-slave-754n"
server_key: "1478ddc58850f1b3f39fdf40e20fb5cf"
server_key_iv: "fd171ed71d0c89dcf249ca130f389e2f"
server_key_version: "encryptionkey@0"
I20260923 23:14:35.841811 28501 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 23:14:35.842509 28516 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:35.842693 28501 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 23:14:35.842815 28501 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790205259943518-25487-0/minicluster-data/master-0/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790205259943518-25487-0/minicluster-data/master-0/wal
uuid: "688c347bae604d06b4fccb39d80e03bc"
format_stamp: "Formatted at 2026-09-23 23:14:35 on dist-test-slave-754n"
server_key: "1478ddc58850f1b3f39fdf40e20fb5cf"
server_key_iv: "fd171ed71d0c89dcf249ca130f389e2f"
server_key_version: "encryptionkey@0"
I20260923 23:14:35.842890 28501 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790205259943518-25487-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790205259943518-25487-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790205259943518-25487-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:14:35.855808 28501 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:14:35.857511 28501 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:14:35.857659 28501 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:14:35.861449 28501 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.254:38139
I20260923 23:14:35.861478 28568 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.254:38139 every 8 connection(s)
I20260923 23:14:35.861814 28501 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
I20260923 23:14:35.862708 28569 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:14:35.864948 28569 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 688c347bae604d06b4fccb39d80e03bc: Bootstrap starting.
I20260923 23:14:35.865535 28569 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 688c347bae604d06b4fccb39d80e03bc: Neither blocks nor log segments found. Creating new log.
I20260923 23:14:35.865811 28569 log.cc:824] T 00000000000000000000000000000000 P 688c347bae604d06b4fccb39d80e03bc: Log is configured to *not* fsync() on all Append() calls
I20260923 23:14:35.866482 28569 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 688c347bae604d06b4fccb39d80e03bc: No bootstrap required, opened a new log
I20260923 23:14:35.868041 28569 raft_consensus.cc:348] T 00000000000000000000000000000000 P 688c347bae604d06b4fccb39d80e03bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "688c347bae604d06b4fccb39d80e03bc" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 38139 } }
I20260923 23:14:35.868170 28569 raft_consensus.cc:374] T 00000000000000000000000000000000 P 688c347bae604d06b4fccb39d80e03bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:14:35.868227 28569 raft_consensus.cc:729] T 00000000000000000000000000000000 P 688c347bae604d06b4fccb39d80e03bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 688c347bae604d06b4fccb39d80e03bc, State: Initialized, Role: FOLLOWER
I20260923 23:14:35.868388 28569 consensus_queue.cc:260] T 00000000000000000000000000000000 P 688c347bae604d06b4fccb39d80e03bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "688c347bae604d06b4fccb39d80e03bc" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 38139 } }
I20260923 23:14:35.868449 28569 raft_consensus.cc:388] T 00000000000000000000000000000000 P 688c347bae604d06b4fccb39d80e03bc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 23:14:35.868470 28569 raft_consensus.cc:482] T 00000000000000000000000000000000 P 688c347bae604d06b4fccb39d80e03bc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 23:14:35.868531 28569 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 688c347bae604d06b4fccb39d80e03bc [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:14:35.868584 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 28501
I20260923 23:14:35.868707 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790205259943518-25487-0/minicluster-data/master-0/wal/instance
I20260923 23:14:35.868893 25487 external_mini_cluster.cc:1428] Setting key 3e52f7efa27adb99d9b5f56ac8259fe5
I20260923 23:14:35.869549 28569 raft_consensus.cc:504] T 00000000000000000000000000000000 P 688c347bae604d06b4fccb39d80e03bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "688c347bae604d06b4fccb39d80e03bc" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 38139 } }
I20260923 23:14:35.869671 28569 leader_election.cc:302] T 00000000000000000000000000000000 P 688c347bae604d06b4fccb39d80e03bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 688c347bae604d06b4fccb39d80e03bc; no voters:
I20260923 23:14:35.869849 28569 leader_election.cc:288] T 00000000000000000000000000000000 P 688c347bae604d06b4fccb39d80e03bc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 23:14:35.869917 28574 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 688c347bae604d06b4fccb39d80e03bc [term 1 FOLLOWER]: Leader election won for term 1
I20260923 23:14:35.870098 28574 raft_consensus.cc:686] T 00000000000000000000000000000000 P 688c347bae604d06b4fccb39d80e03bc [term 1 LEADER]: Becoming Leader. State: Replica: 688c347bae604d06b4fccb39d80e03bc, State: Running, Role: LEADER
I20260923 23:14:35.870152 28569 sys_catalog.cc:564] T 00000000000000000000000000000000 P 688c347bae604d06b4fccb39d80e03bc [sys.catalog]: configured and running, proceeding with master startup.
I20260923 23:14:35.870203 28574 consensus_queue.cc:237] T 00000000000000000000000000000000 P 688c347bae604d06b4fccb39d80e03bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "688c347bae604d06b4fccb39d80e03bc" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 38139 } }
I20260923 23:14:35.870517 28574 sys_catalog.cc:455] T 00000000000000000000000000000000 P 688c347bae604d06b4fccb39d80e03bc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 688c347bae604d06b4fccb39d80e03bc. Latest consensus state: current_term: 1 leader_uuid: "688c347bae604d06b4fccb39d80e03bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "688c347bae604d06b4fccb39d80e03bc" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 38139 } } }
I20260923 23:14:35.870713 28574 sys_catalog.cc:458] T 00000000000000000000000000000000 P 688c347bae604d06b4fccb39d80e03bc [sys.catalog]: This master's current role is: LEADER
I20260923 23:14:35.870645 28575 sys_catalog.cc:455] T 00000000000000000000000000000000 P 688c347bae604d06b4fccb39d80e03bc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "688c347bae604d06b4fccb39d80e03bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "688c347bae604d06b4fccb39d80e03bc" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 38139 } } }
I20260923 23:14:35.871117 28575 sys_catalog.cc:458] T 00000000000000000000000000000000 P 688c347bae604d06b4fccb39d80e03bc [sys.catalog]: This master's current role is: LEADER
I20260923 23:14:35.871119 28582 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 23:14:35.871750 28582 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 23:14:35.873347 28582 catalog_manager.cc:1347] Generated new cluster ID: 377d015700b54852aaf4d8c2b102a44d
I20260923 23:14:35.873411 28582 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 23:14:35 dist-test-slave-754n krb5kdc[28488](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205275, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
I20260923 23:14:35.881824 28573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:35.869349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53700 (local address 127.24.227.254:38139)
0923 23:14:35.869502 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:35.869511 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:35.869546 (+ 35us) server_negotiation.cc:407] Connection header received
0923 23:14:35.869632 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:35.869638 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:35.869702 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:35.869910 (+ 208us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:35.870659 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:35.872342 (+ 1683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:35.874509 (+ 2167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:35.875110 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:35.875221 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:35.877864 (+ 2643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:35.877867 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:35.877868 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:35.877878 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:35.880178 (+ 2300us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:35.880588 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:35.880591 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:35.880591 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:35.880738 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:35.880960 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:35.880961 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:35.880961 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:35.881228 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:35.881375 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:35.881499 (+ 124us) server_negotiation.cc:299] Negotiation successful
0923 23:14:35.881624 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":83,"threads_started":1}
I20260923 23:14:35.885686 28582 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 23:14:35.886168 28582 catalog_manager.cc:1504] Loading token signing keys...
I20260923 23:14:35.893647 28582 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 688c347bae604d06b4fccb39d80e03bc: Generated new TSK 0
I20260923 23:14:35.893817 28582 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.227.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.227.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.227.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.193.keytab.
Entry for principal kudu/127.24.227.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.193.keytab.
I20260923 23:14:35.918802 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790205259943518-25487-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790205259943518-25487-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790205259943518-25487-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790205259943518-25487-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.227.193:0
--local_ip_for_outbound_sockets=127.24.227.193
--webserver_interface=127.24.227.193
--webserver_port=0
--tserver_master_addrs=127.24.227.254:38139
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.193.keytab
--principal=kudu/127.24.227.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:46097
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:14:35.994451 28595 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:14:35.994628 28595 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:14:35.994650 28595 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:14:35.995746 28595 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:14:35.995786 28595 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:14:35.995838 28595 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.227.193
I20260923 23:14:35.997299 28595 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:46097
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790205259943518-25487-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790205259943518-25487-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.193.keytab
--principal=kudu/127.24.227.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.227.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790205259943518-25487-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.227.193
--webserver_port=0
--tserver_master_addrs=127.24.227.254:38139
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28595
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.227.193
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790205259943518-25487-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:14:35.997527 28595 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:14:35.997740 28595 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:14:36.002375 28603 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:14:36.002463 28595 server_base.cc:1034] running on GCE node
W20260923 23:14:36.002355 28601 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:36.002345 28600 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:14:36.002784 28595 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:14:36.003031 28595 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:14:36.004164 28595 hybrid_clock.cc:648] HybridClock initialized: now 1790205276004148 us; error 33 us; skew 500 ppm
Sep 23 23:14:36 dist-test-slave-754n krb5kdc[28488](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205276, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:14:36.006649 28595 init.cc:377] Logged in from keytab as kudu/127.24.227.193@KRBTEST.COM (short username kudu)
I20260923 23:14:36.007004 28595 webserver.cc:460] Webserver started at http://127.24.227.193:44445/ using document root <none> and password file <none>
I20260923 23:14:36.007177 28595 fs_manager.cc:359] Metadata directory not provided
I20260923 23:14:36.007216 28595 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:14:36.007306 28595 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:14:36.008282 28595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790205259943518-25487-0/minicluster-data/ts-0/data/instance:
uuid: "b0e35e88865c44fd8d5a2a5a116b4674"
format_stamp: "Formatted at 2026-09-23 23:14:36 on dist-test-slave-754n"
server_key: "6cbfa488f10d18acb3d8bf89d22c2f70"
server_key_iv: "ac5a93c9f7050d48eeda469c38a063ab"
server_key_version: "encryptionkey@0"
I20260923 23:14:36.008551 28595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790205259943518-25487-0/minicluster-data/ts-0/wal/instance:
uuid: "b0e35e88865c44fd8d5a2a5a116b4674"
format_stamp: "Formatted at 2026-09-23 23:14:36 on dist-test-slave-754n"
server_key: "6cbfa488f10d18acb3d8bf89d22c2f70"
server_key_iv: "ac5a93c9f7050d48eeda469c38a063ab"
server_key_version: "encryptionkey@0"
I20260923 23:14:36.009665 28595 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 23:14:36.010272 28610 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:36.010444 28595 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 23:14:36.010537 28595 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790205259943518-25487-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790205259943518-25487-0/minicluster-data/ts-0/wal
uuid: "b0e35e88865c44fd8d5a2a5a116b4674"
format_stamp: "Formatted at 2026-09-23 23:14:36 on dist-test-slave-754n"
server_key: "6cbfa488f10d18acb3d8bf89d22c2f70"
server_key_iv: "ac5a93c9f7050d48eeda469c38a063ab"
server_key_version: "encryptionkey@0"
I20260923 23:14:36.010612 28595 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790205259943518-25487-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790205259943518-25487-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790205259943518-25487-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:14:36.018072 28595 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:14:36.020190 28595 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:14:36.020352 28595 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:14:36.020555 28595 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 23:14:36.020843 28595 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 23:14:36.020897 28595 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:36.020937 28595 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 23:14:36.020964 28595 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:36.026479 28595 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.193:38763
I20260923 23:14:36.026548 28723 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.193:38763 every 8 connection(s)
I20260923 23:14:36.026834 28595 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790205259943518-25487-0/minicluster-data/ts-0/data/info.pb
Sep 23 23:14:36 dist-test-slave-754n krb5kdc[28488](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205276, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.193@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
I20260923 23:14:36.032760 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 28595
I20260923 23:14:36.032886 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790205259943518-25487-0/minicluster-data/ts-0/wal/instance
I20260923 23:14:36.033073 25487 external_mini_cluster.cc:1428] Setting key 46958ea2db27328699f295a3f806055a
I20260923 23:14:36.035482 28573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:36.027525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.193:60141 (local address 127.24.227.254:38139)
0923 23:14:36.027588 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:36.027590 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:36.027735 (+ 145us) server_negotiation.cc:407] Connection header received
0923 23:14:36.028224 (+ 489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:36.028225 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:36.028243 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:36.028293 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:36.029096 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:36.029570 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:36.030479 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:36.030654 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:36.030702 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:36.032613 (+ 1911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:36.032615 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:36.032615 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:36.032621 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:36.034156 (+ 1535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:36.034552 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:36.034554 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:36.034554 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:36.034616 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:36.034856 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:36.034857 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:36.034857 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:36.034968 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:36.035037 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:36.035304 (+ 267us) server_negotiation.cc:299] Negotiation successful
0923 23:14:36.035396 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260923 23:14:36.035612 28726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:36.027485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:38139 (local address 127.24.227.193:60141)
0923 23:14:36.027622 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:36.027659 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:36.028142 (+ 483us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:36.028345 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:36.028348 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:36.028859 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:14:36.029024 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:36.029028 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:36.029646 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:36.029646 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:36.030409 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:36.030411 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:36.030720 (+ 309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:36.030720 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:36.030820 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:36.031191 (+ 371us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:14:36.031198 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:14:36.032517 (+ 1319us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:14:36.034257 (+ 1740us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:36.034263 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:36.034265 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:36.034474 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:36.034682 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:36.034683 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:36.034684 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:36.034782 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:36.035040 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:14:36.035046 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:14:36.035166 (+ 120us) client_negotiation.cc:769] Sending connection context
0923 23:14:36.035270 (+ 104us) client_negotiation.cc:240] Negotiation successful
0923 23:14:36.035410 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":76,"threads_started":1}
I20260923 23:14:36.036033 28724 heartbeater.cc:347] Connected to a master server at 127.24.227.254:38139
I20260923 23:14:36.036116 28724 heartbeater.cc:464] Registering TS with master...
I20260923 23:14:36.036295 28724 heartbeater.cc:511] Master 127.24.227.254:38139 requested a full tablet report, sending...
I20260923 23:14:36.036761 28533 ts_manager.cc:194] Registered new tserver with Master: b0e35e88865c44fd8d5a2a5a116b4674 (127.24.227.193:38763)
I20260923 23:14:36.037386 28533 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.227.193@KRBTEST.COM'} at 127.24.227.193:60141
WARNING: no policy specified for kudu/127.24.227.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.227.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.227.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.194.keytab.
Entry for principal kudu/127.24.227.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.194.keytab.
I20260923 23:14:36.056550 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790205259943518-25487-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790205259943518-25487-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790205259943518-25487-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790205259943518-25487-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.227.194:0
--local_ip_for_outbound_sockets=127.24.227.194
--webserver_interface=127.24.227.194
--webserver_port=0
--tserver_master_addrs=127.24.227.254:38139
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.194.keytab
--principal=kudu/127.24.227.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:46097
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:14:36.131405 28729 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:14:36.131623 28729 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:14:36.131659 28729 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:14:36.132792 28729 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:14:36.132860 28729 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:14:36.132905 28729 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.227.194
I20260923 23:14:36.134298 28729 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:46097
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790205259943518-25487-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790205259943518-25487-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.194.keytab
--principal=kudu/127.24.227.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.227.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790205259943518-25487-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.227.194
--webserver_port=0
--tserver_master_addrs=127.24.227.254:38139
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28729
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.227.194
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790205259943518-25487-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:14:36.134552 28729 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:14:36.134776 28729 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:14:36.139220 28734 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:36.139210 28735 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:36.139209 28737 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:14:36.139400 28729 server_base.cc:1034] running on GCE node
I20260923 23:14:36.139567 28729 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:14:36.139824 28729 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:14:36.140990 28729 hybrid_clock.cc:648] HybridClock initialized: now 1790205276140962 us; error 44 us; skew 500 ppm
Sep 23 23:14:36 dist-test-slave-754n krb5kdc[28488](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205276, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:14:36.143330 28729 init.cc:377] Logged in from keytab as kudu/127.24.227.194@KRBTEST.COM (short username kudu)
I20260923 23:14:36.143759 28729 webserver.cc:460] Webserver started at http://127.24.227.194:41739/ using document root <none> and password file <none>
I20260923 23:14:36.143987 28729 fs_manager.cc:359] Metadata directory not provided
I20260923 23:14:36.144030 28729 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:14:36.144168 28729 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:14:36.145146 28729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790205259943518-25487-0/minicluster-data/ts-1/data/instance:
uuid: "86c963638e82458796660f074de40ff3"
format_stamp: "Formatted at 2026-09-23 23:14:36 on dist-test-slave-754n"
server_key: "108a970cca8b75a1b41fa4bf45d7e770"
server_key_iv: "948a51b60bc50a1ba9aaddff96026b66"
server_key_version: "encryptionkey@0"
I20260923 23:14:36.145495 28729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790205259943518-25487-0/minicluster-data/ts-1/wal/instance:
uuid: "86c963638e82458796660f074de40ff3"
format_stamp: "Formatted at 2026-09-23 23:14:36 on dist-test-slave-754n"
server_key: "108a970cca8b75a1b41fa4bf45d7e770"
server_key_iv: "948a51b60bc50a1ba9aaddff96026b66"
server_key_version: "encryptionkey@0"
I20260923 23:14:36.146802 28729 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 23:14:36.147480 28744 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:36.147640 28729 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 23:14:36.147739 28729 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790205259943518-25487-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790205259943518-25487-0/minicluster-data/ts-1/wal
uuid: "86c963638e82458796660f074de40ff3"
format_stamp: "Formatted at 2026-09-23 23:14:36 on dist-test-slave-754n"
server_key: "108a970cca8b75a1b41fa4bf45d7e770"
server_key_iv: "948a51b60bc50a1ba9aaddff96026b66"
server_key_version: "encryptionkey@0"
I20260923 23:14:36.147830 28729 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790205259943518-25487-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790205259943518-25487-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790205259943518-25487-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:14:36.172080 28729 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:14:36.174204 28729 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:14:36.174358 28729 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:14:36.174563 28729 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 23:14:36.174850 28729 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 23:14:36.174901 28729 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:36.174932 28729 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 23:14:36.174960 28729 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:36.180562 28729 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.194:45265
I20260923 23:14:36.180614 28857 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.194:45265 every 8 connection(s)
I20260923 23:14:36.180914 28729 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790205259943518-25487-0/minicluster-data/ts-1/data/info.pb
Sep 23 23:14:36 dist-test-slave-754n krb5kdc[28488](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205276, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.194@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
I20260923 23:14:36.189291 28573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:36.181563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.194:40589 (local address 127.24.227.254:38139)
0923 23:14:36.181617 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:36.181619 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:36.181821 (+ 202us) server_negotiation.cc:407] Connection header received
0923 23:14:36.182312 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:36.182313 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:36.182324 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:36.182381 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:36.183244 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:36.183748 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:36.184755 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:36.184969 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:36.185057 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:36.187045 (+ 1988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:36.187046 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:36.187046 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:36.187053 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:36.188212 (+ 1159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:36.188550 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:36.188551 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:36.188551 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:36.188618 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:36.188817 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:36.188817 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:36.188817 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:36.188910 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:36.188959 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:36.189175 (+ 216us) server_negotiation.cc:299] Negotiation successful
0923 23:14:36.189233 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260923 23:14:36.189429 28860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:36.181539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:38139 (local address 127.24.227.194:40589)
0923 23:14:36.181710 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:36.181741 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:36.182203 (+ 462us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:36.182442 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:36.182446 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:36.182967 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:14:36.183135 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:36.183139 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:36.183850 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:36.183852 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:36.184659 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:36.184661 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:36.185065 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:36.185066 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:36.185181 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:36.185664 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:14:36.185672 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:14:36.186969 (+ 1297us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:14:36.188287 (+ 1318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:36.188292 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:36.188294 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:36.188487 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:36.188675 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:36.188675 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:36.188676 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:36.188763 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:36.188964 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:14:36.188968 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:14:36.189077 (+ 109us) client_negotiation.cc:769] Sending connection context
0923 23:14:36.189149 (+ 72us) client_negotiation.cc:240] Negotiation successful
0923 23:14:36.189250 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":75,"threads_started":1}
I20260923 23:14:36.189886 28858 heartbeater.cc:347] Connected to a master server at 127.24.227.254:38139
I20260923 23:14:36.189952 28858 heartbeater.cc:464] Registering TS with master...
I20260923 23:14:36.190083 28858 heartbeater.cc:511] Master 127.24.227.254:38139 requested a full tablet report, sending...
I20260923 23:14:36.190404 28533 ts_manager.cc:194] Registered new tserver with Master: 86c963638e82458796660f074de40ff3 (127.24.227.194:45265)
I20260923 23:14:36.190565 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 28729
I20260923 23:14:36.190668 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790205259943518-25487-0/minicluster-data/ts-1/wal/instance
I20260923 23:14:36.190828 25487 external_mini_cluster.cc:1428] Setting key 3aa0bd26e0a15f8b9e358e956ffdcd5a
I20260923 23:14:36.190862 28533 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.227.194@KRBTEST.COM'} at 127.24.227.194:40589
WARNING: no policy specified for kudu/127.24.227.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.227.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.227.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.195.keytab.
Entry for principal kudu/127.24.227.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.195.keytab.
I20260923 23:14:36.213788 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790205259943518-25487-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790205259943518-25487-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790205259943518-25487-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790205259943518-25487-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.227.195:0
--local_ip_for_outbound_sockets=127.24.227.195
--webserver_interface=127.24.227.195
--webserver_port=0
--tserver_master_addrs=127.24.227.254:38139
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.195.keytab
--principal=kudu/127.24.227.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:46097
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:14:36.290227 28863 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:14:36.290400 28863 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:14:36.290421 28863 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:14:36.291468 28863 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:14:36.291501 28863 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:14:36.291527 28863 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.227.195
I20260923 23:14:36.292937 28863 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:46097
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790205259943518-25487-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790205259943518-25487-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.195.keytab
--principal=kudu/127.24.227.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.227.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790205259943518-25487-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.227.195
--webserver_port=0
--tserver_master_addrs=127.24.227.254:38139
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28863
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.227.195
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790205259943518-25487-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:14:36.293107 28863 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:14:36.293264 28863 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260923 23:14:36.297921 28863 server_base.cc:1034] running on GCE node
W20260923 23:14:36.297904 28871 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:36.297906 28869 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:36.297897 28868 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:14:36.298328 28863 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:14:36.298560 28863 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:14:36.299700 28863 hybrid_clock.cc:648] HybridClock initialized: now 1790205276299692 us; error 26 us; skew 500 ppm
Sep 23 23:14:36 dist-test-slave-754n krb5kdc[28488](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205276, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:14:36.302125 28863 init.cc:377] Logged in from keytab as kudu/127.24.227.195@KRBTEST.COM (short username kudu)
I20260923 23:14:36.302477 28863 webserver.cc:460] Webserver started at http://127.24.227.195:37855/ using document root <none> and password file <none>
I20260923 23:14:36.302651 28863 fs_manager.cc:359] Metadata directory not provided
I20260923 23:14:36.302688 28863 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:14:36.302771 28863 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:14:36.303669 28863 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790205259943518-25487-0/minicluster-data/ts-2/data/instance:
uuid: "f3a7bb8d668b4f0dab837bff26fca2a9"
format_stamp: "Formatted at 2026-09-23 23:14:36 on dist-test-slave-754n"
server_key: "dc4cdb5d2c71d493a2701d9fb57fc012"
server_key_iv: "7bd0a3c704211ca75ce55e36f163ec5e"
server_key_version: "encryptionkey@0"
I20260923 23:14:36.304030 28863 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790205259943518-25487-0/minicluster-data/ts-2/wal/instance:
uuid: "f3a7bb8d668b4f0dab837bff26fca2a9"
format_stamp: "Formatted at 2026-09-23 23:14:36 on dist-test-slave-754n"
server_key: "dc4cdb5d2c71d493a2701d9fb57fc012"
server_key_iv: "7bd0a3c704211ca75ce55e36f163ec5e"
server_key_version: "encryptionkey@0"
I20260923 23:14:36.305207 28863 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 23:14:36.305850 28878 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:36.306013 28863 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 23:14:36.306124 28863 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790205259943518-25487-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790205259943518-25487-0/minicluster-data/ts-2/wal
uuid: "f3a7bb8d668b4f0dab837bff26fca2a9"
format_stamp: "Formatted at 2026-09-23 23:14:36 on dist-test-slave-754n"
server_key: "dc4cdb5d2c71d493a2701d9fb57fc012"
server_key_iv: "7bd0a3c704211ca75ce55e36f163ec5e"
server_key_version: "encryptionkey@0"
I20260923 23:14:36.306217 28863 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790205259943518-25487-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790205259943518-25487-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790205259943518-25487-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:14:36.347608 28863 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:14:36.349314 28863 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:14:36.349457 28863 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:14:36.349645 28863 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 23:14:36.349913 28863 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 23:14:36.349962 28863 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:36.349995 28863 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 23:14:36.350020 28863 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:36.355809 28863 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.195:41265
I20260923 23:14:36.355841 28991 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.195:41265 every 8 connection(s)
I20260923 23:14:36.356189 28863 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790205259943518-25487-0/minicluster-data/ts-2/data/info.pb
I20260923 23:14:36.358021 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 28863
I20260923 23:14:36.358098 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790205259943518-25487-0/minicluster-data/ts-2/wal/instance
I20260923 23:14:36.358268 25487 external_mini_cluster.cc:1428] Setting key f666f177065bfeb9885a37b59f55ea38
Sep 23 23:14:36 dist-test-slave-754n krb5kdc[28488](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205276, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.195@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
I20260923 23:14:36.364627 28573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:36.356908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.195:49687 (local address 127.24.227.254:38139)
0923 23:14:36.356948 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:36.356950 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:36.357110 (+ 160us) server_negotiation.cc:407] Connection header received
0923 23:14:36.357555 (+ 445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:36.357556 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:36.357564 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:36.357616 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:36.358588 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:36.359335 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:36.360282 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:36.360438 (+ 156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:36.360482 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:36.362339 (+ 1857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:36.362340 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:36.362340 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:36.362346 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:36.363677 (+ 1331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:36.364032 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:36.364033 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:36.364033 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:36.364077 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:36.364250 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:36.364250 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:36.364250 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:36.364332 (+ 82us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:36.364368 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:36.364527 (+ 159us) server_negotiation.cc:299] Negotiation successful
0923 23:14:36.364574 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260923 23:14:36.364776 28994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:36.356865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:38139 (local address 127.24.227.195:49687)
0923 23:14:36.357004 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:36.357040 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:36.357478 (+ 438us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:36.357662 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:36.357665 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:36.358249 (+ 584us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:14:36.358497 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:36.358503 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:36.359406 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:36.359407 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:36.360209 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:36.360211 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:36.360498 (+ 287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:36.360499 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:36.360603 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:36.360978 (+ 375us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:14:36.360985 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:14:36.362258 (+ 1273us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:14:36.363739 (+ 1481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:36.363743 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:36.363744 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:36.363983 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:36.364130 (+ 147us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:36.364130 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:36.364130 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:36.364213 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:36.364367 (+ 154us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:14:36.364369 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:14:36.364454 (+ 85us) client_negotiation.cc:769] Sending connection context
0923 23:14:36.364517 (+ 63us) client_negotiation.cc:240] Negotiation successful
0923 23:14:36.364610 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":80,"threads_started":1}
I20260923 23:14:36.365266 28992 heartbeater.cc:347] Connected to a master server at 127.24.227.254:38139
I20260923 23:14:36.365361 28992 heartbeater.cc:464] Registering TS with master...
I20260923 23:14:36.365516 28992 heartbeater.cc:511] Master 127.24.227.254:38139 requested a full tablet report, sending...
I20260923 23:14:36.365849 28533 ts_manager.cc:194] Registered new tserver with Master: f3a7bb8d668b4f0dab837bff26fca2a9 (127.24.227.195:41265)
I20260923 23:14:36.366187 28533 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.227.195@KRBTEST.COM'} at 127.24.227.195:49687
I20260923 23:14:36.369752 25487 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 23 23:14:36 dist-test-slave-754n krb5kdc[28488](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205276, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Sep 23 23:14:36 dist-test-slave-754n krb5kdc[28488](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205276, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
I20260923 23:14:36.390650 28573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:36.384518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53712 (local address 127.24.227.254:38139)
0923 23:14:36.384585 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:36.384587 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:36.384650 (+ 63us) server_negotiation.cc:407] Connection header received
0923 23:14:36.384687 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:36.384688 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:36.384695 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:36.384740 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:36.385206 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:36.385744 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:36.386408 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:36.386570 (+ 162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:36.386622 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:36.388679 (+ 2057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:36.388680 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:36.388680 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:36.388685 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:36.389735 (+ 1050us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:36.390018 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:36.390019 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:36.390019 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:36.390061 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:36.390203 (+ 142us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:36.390203 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:36.390204 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:36.390308 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:36.390377 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:36.390492 (+ 115us) server_negotiation.cc:299] Negotiation successful
0923 23:14:36.390545 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
W20260923 23:14:36.390885 28533 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:53712
I20260923 23:14:36.391290 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 28595
I20260923 23:14:36.395794 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 28729
I20260923 23:14:36.399859 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 28863
I20260923 23:14:36.403699 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 28501
2026-09-23T23:14:36Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (1770 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790205259943518-25487-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 23:14:36 dist-test-slave-754n krb5kdc[29004](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 23:14:36 dist-test-slave-754n krb5kdc[29004](info): set up 2 sockets
Sep 23 23:14:36 dist-test-slave-754n krb5kdc[29004](info): commencing operation
krb5kdc: starting...
W20260923 23:14:37.403682 25487 mini_kdc.cc:121] Time spent starting KDC: real 0.981s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790205259943518-25487-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790205259943518-25487-0/krb5kdc/test-user.keytab.
Sep 23 23:14:37 dist-test-slave-754n krb5kdc[29004](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205277, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T23:14:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T23:14:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.227.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.227.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.227.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab.
Entry for principal kudu/127.24.227.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab.
I20260923 23:14:37.512589 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790205259943518-25487-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790205259943518-25487-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790205259943518-25487-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.254:41261
--webserver_interface=127.24.227.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab
--principal=kudu/127.24.227.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:42571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.227.254:41261
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:14:37.588502 29017 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:14:37.588667 29017 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:14:37.588686 29017 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:14:37.589756 29017 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 23:14:37.589794 29017 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:14:37.589807 29017 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:14:37.589840 29017 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 23:14:37.589869 29017 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 23:14:37.591253 29017 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:42571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790205259943518-25487-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790205259943518-25487-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.227.254:41261
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab
--principal=kudu/127.24.227.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.254:41261
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.227.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29017
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790205259943518-25487-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:14:37.591423 29017 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:14:37.591665 29017 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:14:37.596262 29022 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:14:37.596339 29017 server_base.cc:1034] running on GCE node
W20260923 23:14:37.596215 29023 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:37.596216 29025 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:14:37.596750 29017 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:14:37.597030 29017 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:14:37.598188 29017 hybrid_clock.cc:648] HybridClock initialized: now 1790205277598165 us; error 36 us; skew 500 ppm
Sep 23 23:14:37 dist-test-slave-754n krb5kdc[29004](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205277, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:14:37.600458 29017 init.cc:377] Logged in from keytab as kudu/127.24.227.254@KRBTEST.COM (short username kudu)
I20260923 23:14:37.600881 29017 webserver.cc:460] Webserver started at http://127.24.227.254:34441/ using document root <none> and password file <none>
I20260923 23:14:37.601150 29017 fs_manager.cc:359] Metadata directory not provided
I20260923 23:14:37.601221 29017 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:14:37.601361 29017 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:14:37.602316 29017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790205259943518-25487-0/minicluster-data/master-0/data/instance:
uuid: "ddcbc3aaf0f747ba908e97e3098edf1d"
format_stamp: "Formatted at 2026-09-23 23:14:37 on dist-test-slave-754n"
server_key: "9bb15dbbdbe3791edff49ed4d6fd00f3"
server_key_iv: "4b78e9d69553b77e1d53d14ee3922e31"
server_key_version: "encryptionkey@0"
I20260923 23:14:37.602667 29017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790205259943518-25487-0/minicluster-data/master-0/wal/instance:
uuid: "ddcbc3aaf0f747ba908e97e3098edf1d"
format_stamp: "Formatted at 2026-09-23 23:14:37 on dist-test-slave-754n"
server_key: "9bb15dbbdbe3791edff49ed4d6fd00f3"
server_key_iv: "4b78e9d69553b77e1d53d14ee3922e31"
server_key_version: "encryptionkey@0"
I20260923 23:14:37.603955 29017 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260923 23:14:37.604621 29032 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:37.604784 29017 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 23:14:37.604889 29017 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790205259943518-25487-0/minicluster-data/master-0/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790205259943518-25487-0/minicluster-data/master-0/wal
uuid: "ddcbc3aaf0f747ba908e97e3098edf1d"
format_stamp: "Formatted at 2026-09-23 23:14:37 on dist-test-slave-754n"
server_key: "9bb15dbbdbe3791edff49ed4d6fd00f3"
server_key_iv: "4b78e9d69553b77e1d53d14ee3922e31"
server_key_version: "encryptionkey@0"
I20260923 23:14:37.604969 29017 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790205259943518-25487-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790205259943518-25487-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790205259943518-25487-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:14:37.614657 29017 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:14:37.616850 29017 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:14:37.616998 29017 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:14:37.620659 29017 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.254:41261
I20260923 23:14:37.620698 29084 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.254:41261 every 8 connection(s)
I20260923 23:14:37.621204 29017 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
I20260923 23:14:37.621793 29085 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:14:37.623965 29085 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ddcbc3aaf0f747ba908e97e3098edf1d: Bootstrap starting.
I20260923 23:14:37.624456 29085 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ddcbc3aaf0f747ba908e97e3098edf1d: Neither blocks nor log segments found. Creating new log.
I20260923 23:14:37.624679 29085 log.cc:824] T 00000000000000000000000000000000 P ddcbc3aaf0f747ba908e97e3098edf1d: Log is configured to *not* fsync() on all Append() calls
I20260923 23:14:37.625247 29085 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ddcbc3aaf0f747ba908e97e3098edf1d: No bootstrap required, opened a new log
I20260923 23:14:37.626523 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 29017
I20260923 23:14:37.626622 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790205259943518-25487-0/minicluster-data/master-0/wal/instance
I20260923 23:14:37.626778 25487 external_mini_cluster.cc:1428] Setting key b19b7791f1c95334f5deb4fefcd72ad9
I20260923 23:14:37.626796 29085 raft_consensus.cc:348] T 00000000000000000000000000000000 P ddcbc3aaf0f747ba908e97e3098edf1d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddcbc3aaf0f747ba908e97e3098edf1d" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41261 } }
I20260923 23:14:37.626942 29085 raft_consensus.cc:374] T 00000000000000000000000000000000 P ddcbc3aaf0f747ba908e97e3098edf1d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:14:37.626974 29085 raft_consensus.cc:729] T 00000000000000000000000000000000 P ddcbc3aaf0f747ba908e97e3098edf1d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ddcbc3aaf0f747ba908e97e3098edf1d, State: Initialized, Role: FOLLOWER
I20260923 23:14:37.627110 29085 consensus_queue.cc:260] T 00000000000000000000000000000000 P ddcbc3aaf0f747ba908e97e3098edf1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddcbc3aaf0f747ba908e97e3098edf1d" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41261 } }
I20260923 23:14:37.627214 29085 raft_consensus.cc:388] T 00000000000000000000000000000000 P ddcbc3aaf0f747ba908e97e3098edf1d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 23:14:37.627269 29085 raft_consensus.cc:482] T 00000000000000000000000000000000 P ddcbc3aaf0f747ba908e97e3098edf1d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 23:14:37.627324 29085 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ddcbc3aaf0f747ba908e97e3098edf1d [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:14:37.628002 29085 raft_consensus.cc:504] T 00000000000000000000000000000000 P ddcbc3aaf0f747ba908e97e3098edf1d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddcbc3aaf0f747ba908e97e3098edf1d" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41261 } }
I20260923 23:14:37.628115 29085 leader_election.cc:302] T 00000000000000000000000000000000 P ddcbc3aaf0f747ba908e97e3098edf1d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ddcbc3aaf0f747ba908e97e3098edf1d; no voters:
I20260923 23:14:37.628253 29085 leader_election.cc:288] T 00000000000000000000000000000000 P ddcbc3aaf0f747ba908e97e3098edf1d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 23:14:37.628279 29090 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ddcbc3aaf0f747ba908e97e3098edf1d [term 1 FOLLOWER]: Leader election won for term 1
I20260923 23:14:37.628401 29090 raft_consensus.cc:686] T 00000000000000000000000000000000 P ddcbc3aaf0f747ba908e97e3098edf1d [term 1 LEADER]: Becoming Leader. State: Replica: ddcbc3aaf0f747ba908e97e3098edf1d, State: Running, Role: LEADER
I20260923 23:14:37.628473 29085 sys_catalog.cc:564] T 00000000000000000000000000000000 P ddcbc3aaf0f747ba908e97e3098edf1d [sys.catalog]: configured and running, proceeding with master startup.
I20260923 23:14:37.628571 29090 consensus_queue.cc:237] T 00000000000000000000000000000000 P ddcbc3aaf0f747ba908e97e3098edf1d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddcbc3aaf0f747ba908e97e3098edf1d" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41261 } }
I20260923 23:14:37.628937 29092 sys_catalog.cc:455] T 00000000000000000000000000000000 P ddcbc3aaf0f747ba908e97e3098edf1d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ddcbc3aaf0f747ba908e97e3098edf1d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddcbc3aaf0f747ba908e97e3098edf1d" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41261 } } }
I20260923 23:14:37.629029 29092 sys_catalog.cc:458] T 00000000000000000000000000000000 P ddcbc3aaf0f747ba908e97e3098edf1d [sys.catalog]: This master's current role is: LEADER
I20260923 23:14:37.629144 29092 sys_catalog.cc:455] T 00000000000000000000000000000000 P ddcbc3aaf0f747ba908e97e3098edf1d [sys.catalog]: SysCatalogTable state changed. Reason: New leader ddcbc3aaf0f747ba908e97e3098edf1d. Latest consensus state: current_term: 1 leader_uuid: "ddcbc3aaf0f747ba908e97e3098edf1d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddcbc3aaf0f747ba908e97e3098edf1d" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41261 } } }
I20260923 23:14:37.629184 29092 sys_catalog.cc:458] T 00000000000000000000000000000000 P ddcbc3aaf0f747ba908e97e3098edf1d [sys.catalog]: This master's current role is: LEADER
I20260923 23:14:37.629582 29099 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 23:14:37.630060 29099 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 23:14:37.631795 29099 catalog_manager.cc:1347] Generated new cluster ID: c6c9437dd04c4524b9ebd20ebdff836e
I20260923 23:14:37.631847 29099 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 23:14:37 dist-test-slave-754n krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205277, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
I20260923 23:14:37.637667 29089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:37.627161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42868 (local address 127.24.227.254:41261)
0923 23:14:37.627727 (+ 566us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:37.627733 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:37.627755 (+ 22us) server_negotiation.cc:407] Connection header received
0923 23:14:37.627832 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:37.627836 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:37.627911 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:37.628044 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:37.628715 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:37.630046 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:37.630990 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:37.631554 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:37.631689 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:37.634110 (+ 2421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:37.634114 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:37.634114 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:37.634124 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:37.635982 (+ 1858us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:37.636320 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:37.636322 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:37.636323 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:37.636435 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:37.636602 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:37.636602 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:37.636603 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:37.636847 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:37.636983 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:37.637372 (+ 389us) server_negotiation.cc:299] Negotiation successful
0923 23:14:37.637484 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":526,"thread_start_us":483,"threads_started":1}
I20260923 23:14:37.651638 29099 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 23:14:37.652345 29099 catalog_manager.cc:1504] Loading token signing keys...
I20260923 23:14:37.662292 29099 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ddcbc3aaf0f747ba908e97e3098edf1d: Generated new TSK 0
I20260923 23:14:37.662459 29099 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.227.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.227.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.227.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.193.keytab.
Entry for principal kudu/127.24.227.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.193.keytab.
I20260923 23:14:37.694008 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790205259943518-25487-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790205259943518-25487-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790205259943518-25487-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790205259943518-25487-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.227.193:0
--local_ip_for_outbound_sockets=127.24.227.193
--webserver_interface=127.24.227.193
--webserver_port=0
--tserver_master_addrs=127.24.227.254:41261
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.193.keytab
--principal=kudu/127.24.227.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:42571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:14:37.799691 29111 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:14:37.799853 29111 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:14:37.799925 29111 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:14:37.801012 29111 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:14:37.801070 29111 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:14:37.801123 29111 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.227.193
I20260923 23:14:37.802551 29111 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:42571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790205259943518-25487-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790205259943518-25487-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.193.keytab
--principal=kudu/127.24.227.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.227.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790205259943518-25487-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.227.193
--webserver_port=0
--tserver_master_addrs=127.24.227.254:41261
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29111
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.227.193
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790205259943518-25487-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:14:37.802776 29111 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:14:37.802992 29111 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:14:37.807731 29117 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:37.807714 29116 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:14:37.807739 29111 server_base.cc:1034] running on GCE node
W20260923 23:14:37.807714 29119 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:14:37.808046 29111 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:14:37.808279 29111 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:14:37.809435 29111 hybrid_clock.cc:648] HybridClock initialized: now 1790205277809397 us; error 49 us; skew 500 ppm
Sep 23 23:14:37 dist-test-slave-754n krb5kdc[29004](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205277, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:14:37.811733 29111 init.cc:377] Logged in from keytab as kudu/127.24.227.193@KRBTEST.COM (short username kudu)
I20260923 23:14:37.812208 29111 webserver.cc:460] Webserver started at http://127.24.227.193:38857/ using document root <none> and password file <none>
I20260923 23:14:37.812376 29111 fs_manager.cc:359] Metadata directory not provided
I20260923 23:14:37.812413 29111 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:14:37.812494 29111 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:14:37.813396 29111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790205259943518-25487-0/minicluster-data/ts-0/data/instance:
uuid: "86be9d4419d249cfba52b12c3bda4d61"
format_stamp: "Formatted at 2026-09-23 23:14:37 on dist-test-slave-754n"
server_key: "e5ed2ac8a040de64ed30aebc2e2cef6a"
server_key_iv: "2cbfa6d636cd2d3a03a3d2fc959ea269"
server_key_version: "encryptionkey@0"
I20260923 23:14:37.813689 29111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790205259943518-25487-0/minicluster-data/ts-0/wal/instance:
uuid: "86be9d4419d249cfba52b12c3bda4d61"
format_stamp: "Formatted at 2026-09-23 23:14:37 on dist-test-slave-754n"
server_key: "e5ed2ac8a040de64ed30aebc2e2cef6a"
server_key_iv: "2cbfa6d636cd2d3a03a3d2fc959ea269"
server_key_version: "encryptionkey@0"
I20260923 23:14:37.814832 29111 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260923 23:14:37.815439 29126 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:37.815621 29111 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 23:14:37.815692 29111 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790205259943518-25487-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790205259943518-25487-0/minicluster-data/ts-0/wal
uuid: "86be9d4419d249cfba52b12c3bda4d61"
format_stamp: "Formatted at 2026-09-23 23:14:37 on dist-test-slave-754n"
server_key: "e5ed2ac8a040de64ed30aebc2e2cef6a"
server_key_iv: "2cbfa6d636cd2d3a03a3d2fc959ea269"
server_key_version: "encryptionkey@0"
I20260923 23:14:37.815737 29111 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790205259943518-25487-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790205259943518-25487-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790205259943518-25487-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:14:37.827726 29111 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:14:37.829738 29111 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:14:37.829834 29111 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:14:37.830067 29111 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 23:14:37.830335 29111 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 23:14:37.830363 29111 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:37.830410 29111 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 23:14:37.830432 29111 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:37.835661 29111 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.193:38365
I20260923 23:14:37.835727 29239 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.193:38365 every 8 connection(s)
I20260923 23:14:37.836033 29111 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790205259943518-25487-0/minicluster-data/ts-0/data/info.pb
I20260923 23:14:37.839478 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 29111
I20260923 23:14:37.839586 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790205259943518-25487-0/minicluster-data/ts-0/wal/instance
I20260923 23:14:37.839772 25487 external_mini_cluster.cc:1428] Setting key cfc700e28a6af44ec71a84960406c540
Sep 23 23:14:37 dist-test-slave-754n krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205277, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.193@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
I20260923 23:14:37.845407 29089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:37.836666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.193:33597 (local address 127.24.227.254:41261)
0923 23:14:37.836729 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:37.836731 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:37.836917 (+ 186us) server_negotiation.cc:407] Connection header received
0923 23:14:37.837752 (+ 835us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:37.837753 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:37.837765 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:37.837816 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:37.838335 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:37.838802 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:37.839895 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:37.840120 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:37.840213 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:37.842431 (+ 2218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:37.842433 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:37.842433 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:37.842443 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:37.844033 (+ 1590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:37.844467 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:37.844469 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:37.844469 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:37.844532 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:37.844779 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:37.844780 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:37.844781 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:37.844893 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:37.844967 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:37.845242 (+ 275us) server_negotiation.cc:299] Negotiation successful
0923 23:14:37.845320 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260923 23:14:37.845577 29242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:37.836655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41261 (local address 127.24.227.193:33597)
0923 23:14:37.836803 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:37.836835 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:37.837668 (+ 833us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:37.837863 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:37.837866 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:37.838088 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:14:37.838258 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:37.838261 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:37.838875 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:37.838876 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:37.839765 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:37.839767 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:37.840223 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:37.840224 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:37.840364 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:37.840869 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:14:37.840877 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:14:37.842333 (+ 1456us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:14:37.844147 (+ 1814us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:37.844153 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:37.844155 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:37.844380 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:37.844606 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:37.844606 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:37.844607 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:37.844715 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:37.844958 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:14:37.844964 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:14:37.845116 (+ 152us) client_negotiation.cc:769] Sending connection context
0923 23:14:37.845219 (+ 103us) client_negotiation.cc:240] Negotiation successful
0923 23:14:37.845360 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":93,"threads_started":1}
I20260923 23:14:37.846055 29240 heartbeater.cc:347] Connected to a master server at 127.24.227.254:41261
I20260923 23:14:37.846138 29240 heartbeater.cc:464] Registering TS with master...
I20260923 23:14:37.846309 29240 heartbeater.cc:511] Master 127.24.227.254:41261 requested a full tablet report, sending...
I20260923 23:14:37.846803 29049 ts_manager.cc:194] Registered new tserver with Master: 86be9d4419d249cfba52b12c3bda4d61 (127.24.227.193:38365)
WARNING: no policy specified for kudu/127.24.227.194@KRBTEST.COM; defaulting to no policy
I20260923 23:14:37.847728 29049 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.227.193@KRBTEST.COM'} at 127.24.227.193:33597
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.227.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.227.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.194.keytab.
Entry for principal kudu/127.24.227.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.194.keytab.
I20260923 23:14:37.865955 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790205259943518-25487-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790205259943518-25487-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790205259943518-25487-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790205259943518-25487-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.227.194:0
--local_ip_for_outbound_sockets=127.24.227.194
--webserver_interface=127.24.227.194
--webserver_port=0
--tserver_master_addrs=127.24.227.254:41261
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.194.keytab
--principal=kudu/127.24.227.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:42571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:14:37.942523 29245 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:14:37.942696 29245 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:14:37.942718 29245 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:14:37.943745 29245 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:14:37.943783 29245 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:14:37.943807 29245 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.227.194
I20260923 23:14:37.945215 29245 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:42571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790205259943518-25487-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790205259943518-25487-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.194.keytab
--principal=kudu/127.24.227.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.227.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790205259943518-25487-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.227.194
--webserver_port=0
--tserver_master_addrs=127.24.227.254:41261
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29245
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.227.194
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790205259943518-25487-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:14:37.945377 29245 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:14:37.945523 29245 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:14:37.949884 29251 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:37.949927 29253 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:14:37.950063 29245 server_base.cc:1034] running on GCE node
W20260923 23:14:37.949944 29250 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:14:37.950299 29245 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:14:37.950510 29245 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:14:37.951665 29245 hybrid_clock.cc:648] HybridClock initialized: now 1790205277951648 us; error 32 us; skew 500 ppm
Sep 23 23:14:37 dist-test-slave-754n krb5kdc[29004](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205277, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:14:37.954054 29245 init.cc:377] Logged in from keytab as kudu/127.24.227.194@KRBTEST.COM (short username kudu)
I20260923 23:14:37.954493 29245 webserver.cc:460] Webserver started at http://127.24.227.194:39973/ using document root <none> and password file <none>
I20260923 23:14:37.954667 29245 fs_manager.cc:359] Metadata directory not provided
I20260923 23:14:37.954707 29245 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:14:37.954789 29245 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:14:37.955672 29245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790205259943518-25487-0/minicluster-data/ts-1/data/instance:
uuid: "9cfe85fa8548486992d015f7010b3994"
format_stamp: "Formatted at 2026-09-23 23:14:37 on dist-test-slave-754n"
server_key: "a4288facf3f8a3af24f4074fe9ac919c"
server_key_iv: "6735ba5598aefa9ee35b8be52d14bf5d"
server_key_version: "encryptionkey@0"
I20260923 23:14:37.956020 29245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790205259943518-25487-0/minicluster-data/ts-1/wal/instance:
uuid: "9cfe85fa8548486992d015f7010b3994"
format_stamp: "Formatted at 2026-09-23 23:14:37 on dist-test-slave-754n"
server_key: "a4288facf3f8a3af24f4074fe9ac919c"
server_key_iv: "6735ba5598aefa9ee35b8be52d14bf5d"
server_key_version: "encryptionkey@0"
I20260923 23:14:37.957165 29245 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 23:14:37.957815 29260 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:37.957968 29245 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 23:14:37.958045 29245 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790205259943518-25487-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790205259943518-25487-0/minicluster-data/ts-1/wal
uuid: "9cfe85fa8548486992d015f7010b3994"
format_stamp: "Formatted at 2026-09-23 23:14:37 on dist-test-slave-754n"
server_key: "a4288facf3f8a3af24f4074fe9ac919c"
server_key_iv: "6735ba5598aefa9ee35b8be52d14bf5d"
server_key_version: "encryptionkey@0"
I20260923 23:14:37.958101 29245 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790205259943518-25487-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790205259943518-25487-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790205259943518-25487-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:14:37.967763 29245 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:14:37.969758 29245 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:14:37.969852 29245 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:14:37.969990 29245 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 23:14:37.970191 29245 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 23:14:37.970216 29245 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:37.970232 29245 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 23:14:37.970242 29245 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:37.975335 29245 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.194:45469
I20260923 23:14:37.975399 29373 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.194:45469 every 8 connection(s)
I20260923 23:14:37.975651 29245 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790205259943518-25487-0/minicluster-data/ts-1/data/info.pb
I20260923 23:14:37.979991 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 29245
I20260923 23:14:37.980074 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790205259943518-25487-0/minicluster-data/ts-1/wal/instance
I20260923 23:14:37.980232 25487 external_mini_cluster.cc:1428] Setting key 8e02a586d9d289850ede2d65c386bbb6
Sep 23 23:14:37 dist-test-slave-754n krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205277, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.194@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
I20260923 23:14:37.984369 29089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:37.976313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.194:52971 (local address 127.24.227.254:41261)
0923 23:14:37.976354 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:37.976356 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:37.976542 (+ 186us) server_negotiation.cc:407] Connection header received
0923 23:14:37.977342 (+ 800us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:37.977342 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:37.977351 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:37.977400 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:37.977924 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:37.978386 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:37.979231 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:37.979392 (+ 161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:37.979436 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:37.981539 (+ 2103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:37.981541 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:37.981541 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:37.981548 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:37.982930 (+ 1382us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:37.983437 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:37.983438 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:37.983438 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:37.983491 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:37.983765 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:37.983766 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:37.983766 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:37.983863 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:37.983961 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:37.984248 (+ 287us) server_negotiation.cc:299] Negotiation successful
0923 23:14:37.984310 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260923 23:14:37.984627 29376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:37.976268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41261 (local address 127.24.227.194:52971)
0923 23:14:37.976408 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:37.976443 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:37.977251 (+ 808us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:37.977453 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:37.977474 (+ 21us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:37.977696 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:14:37.977855 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:37.977859 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:37.978455 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:37.978456 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:37.979165 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:37.979167 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:37.979453 (+ 286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:37.979454 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:37.979535 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:37.979958 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:14:37.979966 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:14:37.981448 (+ 1482us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:14:37.983033 (+ 1585us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:37.983040 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:37.983042 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:37.983343 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:37.983559 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:37.983560 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:37.983561 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:37.983702 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:37.983965 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:14:37.983970 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:14:37.984119 (+ 149us) client_negotiation.cc:769] Sending connection context
0923 23:14:37.984227 (+ 108us) client_negotiation.cc:240] Negotiation successful
0923 23:14:37.984386 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":81,"threads_started":1}
I20260923 23:14:37.985147 29374 heartbeater.cc:347] Connected to a master server at 127.24.227.254:41261
I20260923 23:14:37.985234 29374 heartbeater.cc:464] Registering TS with master...
I20260923 23:14:37.985404 29374 heartbeater.cc:511] Master 127.24.227.254:41261 requested a full tablet report, sending...
I20260923 23:14:37.985767 29049 ts_manager.cc:194] Registered new tserver with Master: 9cfe85fa8548486992d015f7010b3994 (127.24.227.194:45469)
I20260923 23:14:37.986143 29049 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.227.194@KRBTEST.COM'} at 127.24.227.194:52971
WARNING: no policy specified for kudu/127.24.227.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.227.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.227.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.195.keytab.
Entry for principal kudu/127.24.227.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.195.keytab.
I20260923 23:14:38.005801 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790205259943518-25487-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790205259943518-25487-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790205259943518-25487-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790205259943518-25487-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.227.195:0
--local_ip_for_outbound_sockets=127.24.227.195
--webserver_interface=127.24.227.195
--webserver_port=0
--tserver_master_addrs=127.24.227.254:41261
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.195.keytab
--principal=kudu/127.24.227.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:42571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:14:38.080214 29379 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:14:38.080404 29379 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:14:38.080436 29379 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:14:38.081538 29379 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:14:38.081604 29379 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:14:38.081645 29379 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.227.195
I20260923 23:14:38.083102 29379 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:42571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790205259943518-25487-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790205259943518-25487-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.195.keytab
--principal=kudu/127.24.227.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.227.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790205259943518-25487-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.227.195
--webserver_port=0
--tserver_master_addrs=127.24.227.254:41261
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29379
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.227.195
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790205259943518-25487-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:14:38.083346 29379 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:14:38.083568 29379 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:14:38.088027 29385 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:38.088047 29384 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:38.088037 29387 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:14:38.088305 29379 server_base.cc:1034] running on GCE node
I20260923 23:14:38.088531 29379 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:14:38.088755 29379 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:14:38.089906 29379 hybrid_clock.cc:648] HybridClock initialized: now 1790205278089902 us; error 33 us; skew 500 ppm
Sep 23 23:14:38 dist-test-slave-754n krb5kdc[29004](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205278, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:14:38.092152 29379 init.cc:377] Logged in from keytab as kudu/127.24.227.195@KRBTEST.COM (short username kudu)
I20260923 23:14:38.092566 29379 webserver.cc:460] Webserver started at http://127.24.227.195:44999/ using document root <none> and password file <none>
I20260923 23:14:38.092797 29379 fs_manager.cc:359] Metadata directory not provided
I20260923 23:14:38.092862 29379 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:14:38.093004 29379 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:14:38.093935 29379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790205259943518-25487-0/minicluster-data/ts-2/data/instance:
uuid: "4b181e4331d94bf7833e9de4ed1ec7cf"
format_stamp: "Formatted at 2026-09-23 23:14:38 on dist-test-slave-754n"
server_key: "f50f8618e759c85f3468fb04ab4ace2f"
server_key_iv: "0c35e7028527dd4ea821c0986de1e8cb"
server_key_version: "encryptionkey@0"
I20260923 23:14:38.094265 29379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790205259943518-25487-0/minicluster-data/ts-2/wal/instance:
uuid: "4b181e4331d94bf7833e9de4ed1ec7cf"
format_stamp: "Formatted at 2026-09-23 23:14:38 on dist-test-slave-754n"
server_key: "f50f8618e759c85f3468fb04ab4ace2f"
server_key_iv: "0c35e7028527dd4ea821c0986de1e8cb"
server_key_version: "encryptionkey@0"
I20260923 23:14:38.095463 29379 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 23:14:38.096151 29394 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:38.096294 29379 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 23:14:38.096379 29379 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790205259943518-25487-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790205259943518-25487-0/minicluster-data/ts-2/wal
uuid: "4b181e4331d94bf7833e9de4ed1ec7cf"
format_stamp: "Formatted at 2026-09-23 23:14:38 on dist-test-slave-754n"
server_key: "f50f8618e759c85f3468fb04ab4ace2f"
server_key_iv: "0c35e7028527dd4ea821c0986de1e8cb"
server_key_version: "encryptionkey@0"
I20260923 23:14:38.096451 29379 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790205259943518-25487-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790205259943518-25487-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790205259943518-25487-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:14:38.107364 29379 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:14:38.109202 29379 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:14:38.109360 29379 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:14:38.109565 29379 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 23:14:38.109865 29379 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 23:14:38.109917 29379 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:38.109956 29379 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 23:14:38.109982 29379 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:38.115335 29379 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.195:38165
I20260923 23:14:38.115403 29507 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.195:38165 every 8 connection(s)
I20260923 23:14:38.115671 29379 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790205259943518-25487-0/minicluster-data/ts-2/data/info.pb
I20260923 23:14:38.119961 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 29379
I20260923 23:14:38.120060 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790205259943518-25487-0/minicluster-data/ts-2/wal/instance
I20260923 23:14:38.120247 25487 external_mini_cluster.cc:1428] Setting key df25ac32cd73e2751e42d12e8160e405
Sep 23 23:14:38 dist-test-slave-754n krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205278, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.195@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
I20260923 23:14:38.124228 29089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:38.116312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.195:50981 (local address 127.24.227.254:41261)
0923 23:14:38.116382 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:38.116384 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:38.116552 (+ 168us) server_negotiation.cc:407] Connection header received
0923 23:14:38.117256 (+ 704us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:38.117257 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:38.117265 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:38.117310 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:38.117795 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:38.118343 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:38.119213 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:38.119428 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:38.119489 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:38.121692 (+ 2203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:38.121693 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:38.121693 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:38.121699 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:38.123101 (+ 1402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:38.123412 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:38.123424 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:38.123424 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:38.123468 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:38.123639 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:38.123639 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:38.123639 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:38.123722 (+ 83us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:38.123767 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:38.124049 (+ 282us) server_negotiation.cc:299] Negotiation successful
0923 23:14:38.124142 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260923 23:14:38.124409 29510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:38.116293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41261 (local address 127.24.227.195:50981)
0923 23:14:38.116451 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:38.116479 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:38.117184 (+ 705us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:38.117350 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:38.117352 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:38.117542 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:14:38.117709 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:38.117714 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:38.118411 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:38.118412 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:38.119151 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:38.119153 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:38.119503 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:38.119504 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:38.119591 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:38.120045 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:14:38.120055 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:14:38.121615 (+ 1560us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:14:38.123167 (+ 1552us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:38.123171 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:38.123173 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:38.123357 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:38.123518 (+ 161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:38.123518 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:38.123518 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:38.123599 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:38.123771 (+ 172us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:14:38.123774 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:14:38.123907 (+ 133us) client_negotiation.cc:769] Sending connection context
0923 23:14:38.124000 (+ 93us) client_negotiation.cc:240] Negotiation successful
0923 23:14:38.124195 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":91,"threads_started":1}
I20260923 23:14:38.124795 29508 heartbeater.cc:347] Connected to a master server at 127.24.227.254:41261
I20260923 23:14:38.124859 29508 heartbeater.cc:464] Registering TS with master...
I20260923 23:14:38.125010 29508 heartbeater.cc:511] Master 127.24.227.254:41261 requested a full tablet report, sending...
I20260923 23:14:38.125352 29049 ts_manager.cc:194] Registered new tserver with Master: 4b181e4331d94bf7833e9de4ed1ec7cf (127.24.227.195:38165)
I20260923 23:14:38.125717 29049 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.227.195@KRBTEST.COM'} at 127.24.227.195:50981
I20260923 23:14:38.131721 25487 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 23 23:14:38 dist-test-slave-754n krb5kdc[29004](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205278, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 23 23:14:38 dist-test-slave-754n krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205278, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.227.193@KRBTEST.COM
I20260923 23:14:38.152419 29515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:38.145316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36306 (local address 127.24.227.193:38365)
0923 23:14:38.145486 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:38.145490 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:38.145500 (+ 10us) server_negotiation.cc:407] Connection header received
0923 23:14:38.145522 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:38.145523 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:38.145536 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:38.145606 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:38.146064 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:38.146761 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:38.147449 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:38.147758 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:38.147808 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:38.149817 (+ 2009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:38.149823 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:38.149824 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:38.149830 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:38.151348 (+ 1518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:38.151647 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:38.151650 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:38.151650 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:38.151707 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:38.151850 (+ 143us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:38.151850 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:38.151850 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:38.152040 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:38.152144 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:38.152227 (+ 83us) server_negotiation.cc:299] Negotiation successful
0923 23:14:38.152288 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":96,"threads_started":1}
Sep 23 23:14:38 dist-test-slave-754n krb5kdc[29004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205278, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
I20260923 23:14:38.159320 29089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:38.153582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42880 (local address 127.24.227.254:41261)
0923 23:14:38.153631 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:38.153633 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:38.153640 (+ 7us) server_negotiation.cc:407] Connection header received
0923 23:14:38.153689 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:38.153690 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:38.153697 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:38.153739 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:38.154147 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:38.154608 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:38.155274 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:38.155431 (+ 157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:38.155474 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:38.157243 (+ 1769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:38.157244 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:38.157244 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:38.157250 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:38.158443 (+ 1193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:38.158696 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:38.158697 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:38.158697 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:38.158741 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:38.158875 (+ 134us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:38.158876 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:38.158876 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:38.158988 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:38.159047 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:38.159126 (+ 79us) server_negotiation.cc:299] Negotiation successful
0923 23:14:38.159200 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
W20260923 23:14:38.159511 29049 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42880
I20260923 23:14:38.159785 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 29111
I20260923 23:14:38.164546 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 29245
I20260923 23:14:38.168633 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 29379
I20260923 23:14:38.172741 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 29017
2026-09-23T23:14:38Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (1768 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790205259943518-25487-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 23:14:38 dist-test-slave-754n krb5kdc[29520](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 23:14:38 dist-test-slave-754n krb5kdc[29520](info): set up 2 sockets
Sep 23 23:14:38 dist-test-slave-754n krb5kdc[29520](info): commencing operation
krb5kdc: starting...
W20260923 23:14:39.179507 25487 mini_kdc.cc:121] Time spent starting KDC: real 0.987s user 0.001s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790205259943518-25487-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790205259943518-25487-0/krb5kdc/test-user.keytab.
Sep 23 23:14:39 dist-test-slave-754n krb5kdc[29520](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205279, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T23:14:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T23:14:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.227.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.227.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.227.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab.
Entry for principal kudu/127.24.227.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab.
I20260923 23:14:39.285843 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790205259943518-25487-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790205259943518-25487-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790205259943518-25487-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.254:44725
--webserver_interface=127.24.227.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab
--principal=kudu/127.24.227.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:41153
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.227.254:44725
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:14:39.361001 29533 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:14:39.361197 29533 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:14:39.361228 29533 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:14:39.362360 29533 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 23:14:39.362421 29533 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:14:39.362445 29533 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:14:39.362465 29533 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 23:14:39.362495 29533 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 23:14:39.363957 29533 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:41153
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790205259943518-25487-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790205259943518-25487-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.227.254:44725
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab
--principal=kudu/127.24.227.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.254:44725
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.227.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29533
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790205259943518-25487-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:14:39.364209 29533 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:14:39.364424 29533 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:14:39.368607 29541 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:39.368629 29538 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:39.368608 29539 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:14:39.368835 29533 server_base.cc:1034] running on GCE node
I20260923 23:14:39.369014 29533 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:14:39.369223 29533 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:14:39.370359 29533 hybrid_clock.cc:648] HybridClock initialized: now 1790205279370346 us; error 27 us; skew 500 ppm
Sep 23 23:14:39 dist-test-slave-754n krb5kdc[29520](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205279, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:14:39.372565 29533 init.cc:377] Logged in from keytab as kudu/127.24.227.254@KRBTEST.COM (short username kudu)
I20260923 23:14:39.372951 29533 fs_manager.cc:359] Metadata directory not provided
I20260923 23:14:39.373020 29533 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:14:39.373157 29533 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:14:39.374101 29533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790205259943518-25487-0/minicluster-data/master-0/data/instance:
uuid: "ecf0fcb657a74edf957248a52b2dce3f"
format_stamp: "Formatted at 2026-09-23 23:14:39 on dist-test-slave-754n"
server_key: "c3dff718212d3d6f81fa34d082b6e13e"
server_key_iv: "ddd8db4b2a81b0137a2400f6d79744e3"
server_key_version: "encryptionkey@0"
I20260923 23:14:39.374428 29533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790205259943518-25487-0/minicluster-data/master-0/wal/instance:
uuid: "ecf0fcb657a74edf957248a52b2dce3f"
format_stamp: "Formatted at 2026-09-23 23:14:39 on dist-test-slave-754n"
server_key: "c3dff718212d3d6f81fa34d082b6e13e"
server_key_iv: "ddd8db4b2a81b0137a2400f6d79744e3"
server_key_version: "encryptionkey@0"
I20260923 23:14:39.375600 29533 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 23:14:39.376384 29547 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:39.376549 29533 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:39.376639 29533 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790205259943518-25487-0/minicluster-data/master-0/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790205259943518-25487-0/minicluster-data/master-0/wal
uuid: "ecf0fcb657a74edf957248a52b2dce3f"
format_stamp: "Formatted at 2026-09-23 23:14:39 on dist-test-slave-754n"
server_key: "c3dff718212d3d6f81fa34d082b6e13e"
server_key_iv: "ddd8db4b2a81b0137a2400f6d79744e3"
server_key_version: "encryptionkey@0"
I20260923 23:14:39.376724 29533 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790205259943518-25487-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790205259943518-25487-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790205259943518-25487-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:14:39.383842 29533 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:14:39.385980 29533 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:14:39.386125 29533 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:14:39.389886 29533 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.254:44725
I20260923 23:14:39.389912 29599 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.254:44725 every 8 connection(s)
I20260923 23:14:39.390170 29533 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
I20260923 23:14:39.390981 29600 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:14:39.393136 29600 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ecf0fcb657a74edf957248a52b2dce3f: Bootstrap starting.
I20260923 23:14:39.393674 29600 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ecf0fcb657a74edf957248a52b2dce3f: Neither blocks nor log segments found. Creating new log.
I20260923 23:14:39.393962 29600 log.cc:824] T 00000000000000000000000000000000 P ecf0fcb657a74edf957248a52b2dce3f: Log is configured to *not* fsync() on all Append() calls
I20260923 23:14:39.394562 29600 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ecf0fcb657a74edf957248a52b2dce3f: No bootstrap required, opened a new log
I20260923 23:14:39.395977 29600 raft_consensus.cc:348] T 00000000000000000000000000000000 P ecf0fcb657a74edf957248a52b2dce3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecf0fcb657a74edf957248a52b2dce3f" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 44725 } }
I20260923 23:14:39.396087 29600 raft_consensus.cc:374] T 00000000000000000000000000000000 P ecf0fcb657a74edf957248a52b2dce3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:14:39.396126 29600 raft_consensus.cc:729] T 00000000000000000000000000000000 P ecf0fcb657a74edf957248a52b2dce3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ecf0fcb657a74edf957248a52b2dce3f, State: Initialized, Role: FOLLOWER
I20260923 23:14:39.396251 29600 consensus_queue.cc:260] T 00000000000000000000000000000000 P ecf0fcb657a74edf957248a52b2dce3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecf0fcb657a74edf957248a52b2dce3f" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 44725 } }
I20260923 23:14:39.396306 29600 raft_consensus.cc:388] T 00000000000000000000000000000000 P ecf0fcb657a74edf957248a52b2dce3f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 23:14:39.396324 29600 raft_consensus.cc:482] T 00000000000000000000000000000000 P ecf0fcb657a74edf957248a52b2dce3f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 23:14:39.396391 29600 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ecf0fcb657a74edf957248a52b2dce3f [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:14:39.397081 29600 raft_consensus.cc:504] T 00000000000000000000000000000000 P ecf0fcb657a74edf957248a52b2dce3f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecf0fcb657a74edf957248a52b2dce3f" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 44725 } }
I20260923 23:14:39.397179 29600 leader_election.cc:302] T 00000000000000000000000000000000 P ecf0fcb657a74edf957248a52b2dce3f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ecf0fcb657a74edf957248a52b2dce3f; no voters:
I20260923 23:14:39.397351 29600 leader_election.cc:288] T 00000000000000000000000000000000 P ecf0fcb657a74edf957248a52b2dce3f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 23:14:39.397368 29603 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ecf0fcb657a74edf957248a52b2dce3f [term 1 FOLLOWER]: Leader election won for term 1
I20260923 23:14:39.397526 29603 raft_consensus.cc:686] T 00000000000000000000000000000000 P ecf0fcb657a74edf957248a52b2dce3f [term 1 LEADER]: Becoming Leader. State: Replica: ecf0fcb657a74edf957248a52b2dce3f, State: Running, Role: LEADER
I20260923 23:14:39.397611 29600 sys_catalog.cc:564] T 00000000000000000000000000000000 P ecf0fcb657a74edf957248a52b2dce3f [sys.catalog]: configured and running, proceeding with master startup.
I20260923 23:14:39.397621 29603 consensus_queue.cc:237] T 00000000000000000000000000000000 P ecf0fcb657a74edf957248a52b2dce3f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecf0fcb657a74edf957248a52b2dce3f" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 44725 } }
I20260923 23:14:39.398034 29605 sys_catalog.cc:455] T 00000000000000000000000000000000 P ecf0fcb657a74edf957248a52b2dce3f [sys.catalog]: SysCatalogTable state changed. Reason: New leader ecf0fcb657a74edf957248a52b2dce3f. Latest consensus state: current_term: 1 leader_uuid: "ecf0fcb657a74edf957248a52b2dce3f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecf0fcb657a74edf957248a52b2dce3f" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 44725 } } }
I20260923 23:14:39.398041 29604 sys_catalog.cc:455] T 00000000000000000000000000000000 P ecf0fcb657a74edf957248a52b2dce3f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ecf0fcb657a74edf957248a52b2dce3f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecf0fcb657a74edf957248a52b2dce3f" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 44725 } } }
I20260923 23:14:39.398135 29605 sys_catalog.cc:458] T 00000000000000000000000000000000 P ecf0fcb657a74edf957248a52b2dce3f [sys.catalog]: This master's current role is: LEADER
I20260923 23:14:39.398144 29604 sys_catalog.cc:458] T 00000000000000000000000000000000 P ecf0fcb657a74edf957248a52b2dce3f [sys.catalog]: This master's current role is: LEADER
I20260923 23:14:39.398491 29611 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 23:14:39.398989 29611 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 23:14:39.400080 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 29533
I20260923 23:14:39.400190 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790205259943518-25487-0/minicluster-data/master-0/wal/instance
I20260923 23:14:39.400362 25487 external_mini_cluster.cc:1428] Setting key e9f5dd320b071745abd01efaa89ccb14
I20260923 23:14:39.400576 29611 catalog_manager.cc:1347] Generated new cluster ID: a693ee7f2ed04f8394439ac201f6afdc
I20260923 23:14:39.400626 29611 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 23:14:39 dist-test-slave-754n krb5kdc[29520](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205279, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
I20260923 23:14:39.411013 29623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:39.400919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37900 (local address 127.24.227.254:44725)
0923 23:14:39.401044 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:39.401053 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:39.401082 (+ 29us) server_negotiation.cc:407] Connection header received
0923 23:14:39.401184 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:39.401190 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:39.401265 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:39.401426 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:39.402102 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:39.403494 (+ 1392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:39.404554 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:39.405171 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:39.405300 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:39.407748 (+ 2448us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:39.407752 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:39.407752 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:39.407761 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:39.409402 (+ 1641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:39.409779 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:39.409781 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:39.409782 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:39.409876 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:39.410100 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:39.410101 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:39.410101 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:39.410351 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:39.410481 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:39.410587 (+ 106us) server_negotiation.cc:299] Negotiation successful
0923 23:14:39.410794 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86,"thread_start_us":58,"threads_started":1}
I20260923 23:14:39.414623 29611 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 23:14:39.415083 29611 catalog_manager.cc:1504] Loading token signing keys...
I20260923 23:14:39.418612 29611 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ecf0fcb657a74edf957248a52b2dce3f: Generated new TSK 0
I20260923 23:14:39.418762 29611 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.227.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.227.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.227.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.193.keytab.
Entry for principal kudu/127.24.227.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.193.keytab.
I20260923 23:14:39.446323 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790205259943518-25487-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790205259943518-25487-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790205259943518-25487-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790205259943518-25487-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.227.193:0
--local_ip_for_outbound_sockets=127.24.227.193
--webserver_interface=127.24.227.193
--webserver_port=0
--tserver_master_addrs=127.24.227.254:44725
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.193.keytab
--principal=kudu/127.24.227.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:41153
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:14:39.526856 29626 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:14:39.527055 29626 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:14:39.527078 29626 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:14:39.528227 29626 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:14:39.528282 29626 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:14:39.528314 29626 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.227.193
I20260923 23:14:39.529614 29626 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:41153
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790205259943518-25487-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790205259943518-25487-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.193.keytab
--principal=kudu/127.24.227.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.227.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790205259943518-25487-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.227.193
--webserver_port=0
--tserver_master_addrs=127.24.227.254:44725
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29626
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.227.193
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790205259943518-25487-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:14:39.529772 29626 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:14:39.529910 29626 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:14:39.534520 29632 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:39.534556 29634 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:39.534531 29631 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:14:39.534693 29626 server_base.cc:1034] running on GCE node
I20260923 23:14:39.534917 29626 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:14:39.535135 29626 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:14:39.536283 29626 hybrid_clock.cc:648] HybridClock initialized: now 1790205279536278 us; error 32 us; skew 500 ppm
Sep 23 23:14:39 dist-test-slave-754n krb5kdc[29520](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205279, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:14:39.538513 29626 init.cc:377] Logged in from keytab as kudu/127.24.227.193@KRBTEST.COM (short username kudu)
I20260923 23:14:39.538760 29626 fs_manager.cc:359] Metadata directory not provided
I20260923 23:14:39.538797 29626 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:14:39.538883 29626 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:14:39.539708 29626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790205259943518-25487-0/minicluster-data/ts-0/data/instance:
uuid: "9c3e4de1504446d291f74acb1a2b7e1a"
format_stamp: "Formatted at 2026-09-23 23:14:39 on dist-test-slave-754n"
server_key: "ddba3ea15908abe5dc41477b31899e31"
server_key_iv: "68e2146bf537c1a1ab7b6a0f014922e8"
server_key_version: "encryptionkey@0"
I20260923 23:14:39.540071 29626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790205259943518-25487-0/minicluster-data/ts-0/wal/instance:
uuid: "9c3e4de1504446d291f74acb1a2b7e1a"
format_stamp: "Formatted at 2026-09-23 23:14:39 on dist-test-slave-754n"
server_key: "ddba3ea15908abe5dc41477b31899e31"
server_key_iv: "68e2146bf537c1a1ab7b6a0f014922e8"
server_key_version: "encryptionkey@0"
I20260923 23:14:39.541211 29626 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 23:14:39.541914 29640 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:39.542080 29626 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 23:14:39.542166 29626 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790205259943518-25487-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790205259943518-25487-0/minicluster-data/ts-0/wal
uuid: "9c3e4de1504446d291f74acb1a2b7e1a"
format_stamp: "Formatted at 2026-09-23 23:14:39 on dist-test-slave-754n"
server_key: "ddba3ea15908abe5dc41477b31899e31"
server_key_iv: "68e2146bf537c1a1ab7b6a0f014922e8"
server_key_version: "encryptionkey@0"
I20260923 23:14:39.542237 29626 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790205259943518-25487-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790205259943518-25487-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790205259943518-25487-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:14:39.560721 29626 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:14:39.562894 29626 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:14:39.563050 29626 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:14:39.563261 29626 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 23:14:39.563556 29626 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 23:14:39.563608 29626 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:39.563642 29626 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 23:14:39.563668 29626 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:39.569438 29626 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.193:37057
I20260923 23:14:39.569509 29753 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.193:37057 every 8 connection(s)
I20260923 23:14:39.569737 29626 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790205259943518-25487-0/minicluster-data/ts-0/data/info.pb
I20260923 23:14:39.570657 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 29626
I20260923 23:14:39.570750 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790205259943518-25487-0/minicluster-data/ts-0/wal/instance
I20260923 23:14:39.570920 25487 external_mini_cluster.cc:1428] Setting key f790148b732281cff66b6d511ba3b41b
Sep 23 23:14:39 dist-test-slave-754n krb5kdc[29520](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205279, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.193@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.24.227.194@KRBTEST.COM; defaulting to no policy
I20260923 23:14:39.579334 29623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:39.570394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.193:38425 (local address 127.24.227.254:44725)
0923 23:14:39.570434 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:39.570436 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:39.570595 (+ 159us) server_negotiation.cc:407] Connection header received
0923 23:14:39.571129 (+ 534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:39.571130 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:39.571154 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:39.571209 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:39.572162 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:39.572946 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:39.574218 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:39.574418 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:39.574482 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:39.576460 (+ 1978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:39.576461 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:39.576461 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:39.576468 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:39.577921 (+ 1453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:39.578315 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:39.578316 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:39.578317 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:39.578392 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:39.578650 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:39.578651 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:39.578652 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:39.578764 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:39.578835 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:39.579165 (+ 330us) server_negotiation.cc:299] Negotiation successful
0923 23:14:39.579245 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260923 23:14:39.579465 29756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:39.570346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:44725 (local address 127.24.227.193:38425)
0923 23:14:39.570479 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:39.570514 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:39.571002 (+ 488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:39.571263 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:39.571266 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:39.571559 (+ 293us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:14:39.572076 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:39.572081 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:39.573035 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:39.573037 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:39.574138 (+ 1101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:39.574140 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:39.574497 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:39.574498 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:39.574610 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:39.575048 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:14:39.575055 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:14:39.576363 (+ 1308us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:14:39.578015 (+ 1652us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:39.578021 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:39.578023 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:39.578238 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:39.578459 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:39.578460 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:39.578460 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:39.578587 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:39.578845 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:14:39.578852 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:14:39.579020 (+ 168us) client_negotiation.cc:769] Sending connection context
0923 23:14:39.579131 (+ 111us) client_negotiation.cc:240] Negotiation successful
0923 23:14:39.579271 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":75,"threads_started":1}
I20260923 23:14:39.579901 29754 heartbeater.cc:347] Connected to a master server at 127.24.227.254:44725
I20260923 23:14:39.579993 29754 heartbeater.cc:464] Registering TS with master...
I20260923 23:14:39.580114 29754 heartbeater.cc:511] Master 127.24.227.254:44725 requested a full tablet report, sending...
I20260923 23:14:39.580559 29564 ts_manager.cc:194] Registered new tserver with Master: 9c3e4de1504446d291f74acb1a2b7e1a (127.24.227.193:37057)
I20260923 23:14:39.581094 29564 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.227.193@KRBTEST.COM'} at 127.24.227.193:38425
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.227.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.227.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.194.keytab.
Entry for principal kudu/127.24.227.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.194.keytab.
I20260923 23:14:39.594192 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790205259943518-25487-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790205259943518-25487-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790205259943518-25487-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790205259943518-25487-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.227.194:0
--local_ip_for_outbound_sockets=127.24.227.194
--webserver_interface=127.24.227.194
--webserver_port=0
--tserver_master_addrs=127.24.227.254:44725
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.194.keytab
--principal=kudu/127.24.227.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:41153
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:14:39.667706 29759 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:14:39.667917 29759 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:14:39.667964 29759 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:14:39.669065 29759 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:14:39.669127 29759 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:14:39.669169 29759 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.227.194
I20260923 23:14:39.670578 29759 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:41153
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790205259943518-25487-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790205259943518-25487-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.194.keytab
--principal=kudu/127.24.227.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.227.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790205259943518-25487-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.227.194
--webserver_port=0
--tserver_master_addrs=127.24.227.254:44725
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29759
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.227.194
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790205259943518-25487-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:14:39.670795 29759 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:14:39.670995 29759 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:14:39.675446 29767 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:39.675468 29764 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:14:39.675570 29759 server_base.cc:1034] running on GCE node
W20260923 23:14:39.675468 29765 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:14:39.675812 29759 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:14:39.676069 29759 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:14:39.677239 29759 hybrid_clock.cc:648] HybridClock initialized: now 1790205279677225 us; error 39 us; skew 500 ppm
Sep 23 23:14:39 dist-test-slave-754n krb5kdc[29520](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205279, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:14:39.679772 29759 init.cc:377] Logged in from keytab as kudu/127.24.227.194@KRBTEST.COM (short username kudu)
I20260923 23:14:39.680176 29759 fs_manager.cc:359] Metadata directory not provided
I20260923 23:14:39.680238 29759 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:14:39.680377 29759 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:14:39.681242 29759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790205259943518-25487-0/minicluster-data/ts-1/data/instance:
uuid: "ffafeada02ea4eeda4d0f8f7d098d485"
format_stamp: "Formatted at 2026-09-23 23:14:39 on dist-test-slave-754n"
server_key: "e16195d22eb846773a5c06d6fb05d4f7"
server_key_iv: "d470e5a2ba3060f48df3fab9200182b5"
server_key_version: "encryptionkey@0"
I20260923 23:14:39.681623 29759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790205259943518-25487-0/minicluster-data/ts-1/wal/instance:
uuid: "ffafeada02ea4eeda4d0f8f7d098d485"
format_stamp: "Formatted at 2026-09-23 23:14:39 on dist-test-slave-754n"
server_key: "e16195d22eb846773a5c06d6fb05d4f7"
server_key_iv: "d470e5a2ba3060f48df3fab9200182b5"
server_key_version: "encryptionkey@0"
I20260923 23:14:39.682799 29759 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 23:14:39.683437 29773 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:39.683627 29759 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:39.683719 29759 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790205259943518-25487-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790205259943518-25487-0/minicluster-data/ts-1/wal
uuid: "ffafeada02ea4eeda4d0f8f7d098d485"
format_stamp: "Formatted at 2026-09-23 23:14:39 on dist-test-slave-754n"
server_key: "e16195d22eb846773a5c06d6fb05d4f7"
server_key_iv: "d470e5a2ba3060f48df3fab9200182b5"
server_key_version: "encryptionkey@0"
I20260923 23:14:39.683815 29759 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790205259943518-25487-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790205259943518-25487-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790205259943518-25487-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:14:39.692050 29759 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:14:39.693647 29759 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:14:39.693802 29759 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:14:39.694010 29759 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 23:14:39.694276 29759 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 23:14:39.694370 29759 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:39.694408 29759 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 23:14:39.694442 29759 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:39.700179 29759 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.194:35193
I20260923 23:14:39.700212 29886 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.194:35193 every 8 connection(s)
I20260923 23:14:39.700433 29759 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790205259943518-25487-0/minicluster-data/ts-1/data/info.pb
Sep 23 23:14:39 dist-test-slave-754n krb5kdc[29520](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205279, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.194@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
I20260923 23:14:39.708063 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 29759
I20260923 23:14:39.708197 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790205259943518-25487-0/minicluster-data/ts-1/wal/instance
I20260923 23:14:39.708360 29623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:39.701038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.194:45269 (local address 127.24.227.254:44725)
0923 23:14:39.701103 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:39.701105 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:39.701248 (+ 143us) server_negotiation.cc:407] Connection header received
0923 23:14:39.701677 (+ 429us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:39.701678 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:39.701687 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:39.701739 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:39.702534 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:39.703016 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:39.703987 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:39.704152 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:39.704201 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:39.706073 (+ 1872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:39.706075 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:39.706075 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:39.706081 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:39.707176 (+ 1095us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:39.707497 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:39.707497 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:39.707498 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:39.707541 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:39.707692 (+ 151us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:39.707693 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:39.707693 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:39.707776 (+ 83us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:39.707819 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:39.708209 (+ 390us) server_negotiation.cc:299] Negotiation successful
0923 23:14:39.708288 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260923 23:14:39.708372 25487 external_mini_cluster.cc:1428] Setting key cb4bbff804926c5d10762cfcd12ffedd
I20260923 23:14:39.708561 29889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:39.701011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:44725 (local address 127.24.227.194:45269)
0923 23:14:39.701138 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:39.701172 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:39.701597 (+ 425us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:39.701783 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:39.701786 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:39.702010 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:14:39.702452 (+ 442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:39.702456 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:39.703091 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:39.703091 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:39.703911 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:39.703914 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:39.704306 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:39.704308 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:39.704454 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:39.704817 (+ 363us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:14:39.704824 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:14:39.705986 (+ 1162us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:14:39.707237 (+ 1251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:39.707242 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:39.707243 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:39.707438 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:39.707578 (+ 140us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:39.707578 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:39.707578 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:39.707650 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:39.707827 (+ 177us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:14:39.707831 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:14:39.708028 (+ 197us) client_negotiation.cc:769] Sending connection context
0923 23:14:39.708175 (+ 147us) client_negotiation.cc:240] Negotiation successful
0923 23:14:39.708330 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":70,"threads_started":1}
I20260923 23:14:39.709048 29887 heartbeater.cc:347] Connected to a master server at 127.24.227.254:44725
I20260923 23:14:39.709147 29887 heartbeater.cc:464] Registering TS with master...
I20260923 23:14:39.709295 29887 heartbeater.cc:511] Master 127.24.227.254:44725 requested a full tablet report, sending...
I20260923 23:14:39.709682 29563 ts_manager.cc:194] Registered new tserver with Master: ffafeada02ea4eeda4d0f8f7d098d485 (127.24.227.194:35193)
I20260923 23:14:39.710040 29563 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.227.194@KRBTEST.COM'} at 127.24.227.194:45269
WARNING: no policy specified for kudu/127.24.227.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.227.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.227.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.195.keytab.
Entry for principal kudu/127.24.227.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.195.keytab.
I20260923 23:14:39.730726 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790205259943518-25487-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790205259943518-25487-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790205259943518-25487-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790205259943518-25487-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.227.195:0
--local_ip_for_outbound_sockets=127.24.227.195
--webserver_interface=127.24.227.195
--webserver_port=0
--tserver_master_addrs=127.24.227.254:44725
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.195.keytab
--principal=kudu/127.24.227.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:41153
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:14:39.808107 29892 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:14:39.808306 29892 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:14:39.808338 29892 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:14:39.809401 29892 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:14:39.809474 29892 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:14:39.809515 29892 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.227.195
I20260923 23:14:39.810971 29892 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:41153
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790205259943518-25487-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790205259943518-25487-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.195.keytab
--principal=kudu/127.24.227.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.227.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790205259943518-25487-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.227.195
--webserver_port=0
--tserver_master_addrs=127.24.227.254:44725
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29892
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.227.195
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790205259943518-25487-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:14:39.811225 29892 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:14:39.811496 29892 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:14:39.816087 29898 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:39.816075 29897 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:39.816075 29900 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:14:39.816298 29892 server_base.cc:1034] running on GCE node
I20260923 23:14:39.816516 29892 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:14:39.816737 29892 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:14:39.817912 29892 hybrid_clock.cc:648] HybridClock initialized: now 1790205279817906 us; error 35 us; skew 500 ppm
Sep 23 23:14:39 dist-test-slave-754n krb5kdc[29520](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205279, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:14:39.820340 29892 init.cc:377] Logged in from keytab as kudu/127.24.227.195@KRBTEST.COM (short username kudu)
I20260923 23:14:39.820638 29892 fs_manager.cc:359] Metadata directory not provided
I20260923 23:14:39.820698 29892 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:14:39.820822 29892 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:14:39.821777 29892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790205259943518-25487-0/minicluster-data/ts-2/data/instance:
uuid: "f692378723c742f0a9c65df33d201a6d"
format_stamp: "Formatted at 2026-09-23 23:14:39 on dist-test-slave-754n"
server_key: "dc73fb314a063fddd9e6c903e9af59de"
server_key_iv: "f05244776508a80d43446c828b30ab5c"
server_key_version: "encryptionkey@0"
I20260923 23:14:39.822134 29892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790205259943518-25487-0/minicluster-data/ts-2/wal/instance:
uuid: "f692378723c742f0a9c65df33d201a6d"
format_stamp: "Formatted at 2026-09-23 23:14:39 on dist-test-slave-754n"
server_key: "dc73fb314a063fddd9e6c903e9af59de"
server_key_iv: "f05244776508a80d43446c828b30ab5c"
server_key_version: "encryptionkey@0"
I20260923 23:14:39.823354 29892 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260923 23:14:39.824088 29906 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:39.824250 29892 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 23:14:39.824329 29892 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790205259943518-25487-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790205259943518-25487-0/minicluster-data/ts-2/wal
uuid: "f692378723c742f0a9c65df33d201a6d"
format_stamp: "Formatted at 2026-09-23 23:14:39 on dist-test-slave-754n"
server_key: "dc73fb314a063fddd9e6c903e9af59de"
server_key_iv: "f05244776508a80d43446c828b30ab5c"
server_key_version: "encryptionkey@0"
I20260923 23:14:39.824409 29892 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790205259943518-25487-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790205259943518-25487-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790205259943518-25487-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:14:39.838798 29892 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:14:39.840636 29892 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:14:39.840780 29892 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:14:39.840988 29892 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 23:14:39.841277 29892 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 23:14:39.841364 29892 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:39.841441 29892 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 23:14:39.841471 29892 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:39.847119 29892 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.195:38337
I20260923 23:14:39.847186 30019 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.195:38337 every 8 connection(s)
I20260923 23:14:39.847430 29892 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790205259943518-25487-0/minicluster-data/ts-2/data/info.pb
Sep 23 23:14:39 dist-test-slave-754n krb5kdc[29520](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205279, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.195@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
I20260923 23:14:39.854657 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 29892
I20260923 23:14:39.854792 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790205259943518-25487-0/minicluster-data/ts-2/wal/instance
I20260923 23:14:39.854971 25487 external_mini_cluster.cc:1428] Setting key f659d11b602c15f7f3cce329c38573f4
I20260923 23:14:39.855705 29623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:39.848104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.195:56235 (local address 127.24.227.254:44725)
0923 23:14:39.848142 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:39.848144 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:39.848319 (+ 175us) server_negotiation.cc:407] Connection header received
0923 23:14:39.848733 (+ 414us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:39.848734 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:39.848742 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:39.848798 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:39.849561 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:39.850003 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:39.850865 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:39.851033 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:39.851080 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:39.853021 (+ 1941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:39.853023 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:39.853023 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:39.853030 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:39.854397 (+ 1367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:39.854728 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:39.854729 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:39.854729 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:39.854794 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:39.855054 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:39.855055 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:39.855056 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:39.855222 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:39.855281 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:39.855545 (+ 264us) server_negotiation.cc:299] Negotiation successful
0923 23:14:39.855621 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260923 23:14:39.855904 30022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:39.848059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:44725 (local address 127.24.227.195:56235)
0923 23:14:39.848208 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:39.848245 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:39.848659 (+ 414us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:39.848844 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:39.848848 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:39.849064 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:14:39.849481 (+ 417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:39.849485 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:39.850075 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:39.850076 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:39.850794 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:39.850795 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:39.851099 (+ 304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:39.851099 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:39.851189 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:39.851581 (+ 392us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:14:39.851588 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:14:39.852912 (+ 1324us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:14:39.854463 (+ 1551us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:39.854467 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:39.854469 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:39.854649 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:39.854874 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:39.854875 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:39.854875 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:39.854979 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:39.855281 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:14:39.855288 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:14:39.855417 (+ 129us) client_negotiation.cc:769] Sending connection context
0923 23:14:39.855521 (+ 104us) client_negotiation.cc:240] Negotiation successful
0923 23:14:39.855655 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":87,"threads_started":1}
I20260923 23:14:39.856272 30020 heartbeater.cc:347] Connected to a master server at 127.24.227.254:44725
I20260923 23:14:39.856346 30020 heartbeater.cc:464] Registering TS with master...
I20260923 23:14:39.856505 30020 heartbeater.cc:511] Master 127.24.227.254:44725 requested a full tablet report, sending...
I20260923 23:14:39.856858 29563 ts_manager.cc:194] Registered new tserver with Master: f692378723c742f0a9c65df33d201a6d (127.24.227.195:38337)
I20260923 23:14:39.857223 29563 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.227.195@KRBTEST.COM'} at 127.24.227.195:56235
I20260923 23:14:39.866544 25487 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 23:14:39.873574 29623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:39.868063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37914 (local address 127.24.227.254:44725)
0923 23:14:39.868162 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:39.868164 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:39.868233 (+ 69us) server_negotiation.cc:407] Connection header received
0923 23:14:39.868308 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:39.868308 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:39.868315 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:39.868360 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:39.868856 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:39.869287 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:39.869986 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:39.870159 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:39.870216 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:39.871251 (+ 1035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:39.871252 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:39.871253 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:39.871258 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:39.872662 (+ 1404us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:39.872982 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:39.872982 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:39.872983 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:39.873028 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:39.873172 (+ 144us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:39.873172 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:39.873172 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:39.873280 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:39.873343 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:39.873442 (+ 99us) server_negotiation.cc:299] Negotiation successful
0923 23:14:39.873494 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260923 23:14:39.875074 29563 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37914:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 23:14:39.876586 29563 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 23:14:39.882966 30032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:39.878802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52802 (local address 127.24.227.195:38337)
0923 23:14:39.878970 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:39.878974 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:39.879340 (+ 366us) server_negotiation.cc:407] Connection header received
0923 23:14:39.879459 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:39.879462 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:39.879496 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:39.879565 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:14:39.879744 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:39.880785 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:39.881882 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:39.882583 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:39.882664 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:39.882716 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:39.882851 (+ 135us) server_negotiation.cc:299] Negotiation successful
0923 23:14:39.882889 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":88,"threads_started":1}
I20260923 23:14:39.882961 30029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:39.878608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.195:38337 (local address 127.0.0.1:52802)
0923 23:14:39.879287 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:39.879295 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:39.879363 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:39.879634 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:39.879635 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:39.879637 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:14:39.879672 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:39.879673 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:39.880867 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:39.880868 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:39.881801 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:39.881803 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:39.882669 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:39.882670 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:39.882722 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:39.882732 (+ 10us) client_negotiation.cc:769] Sending connection context
0923 23:14:39.882829 (+ 97us) client_negotiation.cc:240] Negotiation successful
0923 23:14:39.882884 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":669,"thread_start_us":74,"threads_started":1}
I20260923 23:14:39.883358 30030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:39.878597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.194:35193 (local address 127.0.0.1:37476)
0923 23:14:39.879126 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:39.879135 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:39.879206 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:39.879454 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:39.879454 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:39.879456 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:14:39.879502 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:39.879503 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:39.880593 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:39.880594 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:39.881949 (+ 1355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:39.881952 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:39.882803 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:39.882804 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:39.882857 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:39.882860 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 23:14:39.882918 (+ 58us) client_negotiation.cc:240] Negotiation successful
0923 23:14:39.882950 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":514,"mutex_wait_us":32,"thread_start_us":52,"threads_started":1}
I20260923 23:14:39.883625 30031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:39.878778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37476 (local address 127.24.227.194:35193)
0923 23:14:39.878923 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:39.878927 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:39.879202 (+ 275us) server_negotiation.cc:407] Connection header received
0923 23:14:39.879274 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:39.879277 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:39.879294 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:39.879366 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:14:39.879577 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:39.880508 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:39.882034 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:39.882716 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:39.882782 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:39.882828 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:39.883488 (+ 660us) server_negotiation.cc:299] Negotiation successful
0923 23:14:39.883538 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":88,"threads_started":1}
I20260923 23:14:39.883683 29954 tablet_service.cc:1467] Processing CreateTablet for tablet 4eadff6c00ec4c4aadab2148d0e99be6 (DEFAULT_TABLE table=test-table [id=0c2c5152cd694301af451d0badce0277]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 23:14:39.884061 29954 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4eadff6c00ec4c4aadab2148d0e99be6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:14:39.884176 30028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:39.878597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.193:37057 (local address 127.0.0.1:39012)
0923 23:14:39.878734 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:39.878788 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:39.878852 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:39.879285 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:39.879288 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:39.879290 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:14:39.879351 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:39.879352 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:39.880675 (+ 1323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:39.880675 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:39.881955 (+ 1280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:39.881956 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:39.883657 (+ 1701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:39.883658 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:39.883996 (+ 338us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:39.884002 (+ 6us) client_negotiation.cc:769] Sending connection context
0923 23:14:39.884054 (+ 52us) client_negotiation.cc:240] Negotiation successful
0923 23:14:39.884096 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"mutex_wait_us":46,"thread_start_us":48,"threads_started":1}
I20260923 23:14:39.884300 29821 tablet_service.cc:1467] Processing CreateTablet for tablet 4eadff6c00ec4c4aadab2148d0e99be6 (DEFAULT_TABLE table=test-table [id=0c2c5152cd694301af451d0badce0277]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 23:14:39.884415 30033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:39.878990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39012 (local address 127.24.227.193:37057)
0923 23:14:39.879136 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:39.879140 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:39.879147 (+ 7us) server_negotiation.cc:407] Connection header received
0923 23:14:39.879179 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:39.879181 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:39.879192 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:39.879279 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:14:39.879410 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:39.880571 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:39.882863 (+ 2292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:39.883568 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:39.883645 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:39.883694 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:39.884277 (+ 583us) server_negotiation.cc:299] Negotiation successful
0923 23:14:39.884328 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":75,"threads_started":1}
I20260923 23:14:39.884603 29821 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4eadff6c00ec4c4aadab2148d0e99be6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:14:39.887315 30034 tablet_bootstrap.cc:492] T 4eadff6c00ec4c4aadab2148d0e99be6 P f692378723c742f0a9c65df33d201a6d: Bootstrap starting.
I20260923 23:14:39.887764 30035 tablet_bootstrap.cc:492] T 4eadff6c00ec4c4aadab2148d0e99be6 P ffafeada02ea4eeda4d0f8f7d098d485: Bootstrap starting.
I20260923 23:14:39.888022 30034 tablet_bootstrap.cc:654] T 4eadff6c00ec4c4aadab2148d0e99be6 P f692378723c742f0a9c65df33d201a6d: Neither blocks nor log segments found. Creating new log.
I20260923 23:14:39.888432 30035 tablet_bootstrap.cc:654] T 4eadff6c00ec4c4aadab2148d0e99be6 P ffafeada02ea4eeda4d0f8f7d098d485: Neither blocks nor log segments found. Creating new log.
I20260923 23:14:39.888417 29688 tablet_service.cc:1467] Processing CreateTablet for tablet 4eadff6c00ec4c4aadab2148d0e99be6 (DEFAULT_TABLE table=test-table [id=0c2c5152cd694301af451d0badce0277]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 23:14:39.888587 30034 log.cc:824] T 4eadff6c00ec4c4aadab2148d0e99be6 P f692378723c742f0a9c65df33d201a6d: Log is configured to *not* fsync() on all Append() calls
I20260923 23:14:39.888751 29688 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4eadff6c00ec4c4aadab2148d0e99be6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:14:39.888917 30035 log.cc:824] T 4eadff6c00ec4c4aadab2148d0e99be6 P ffafeada02ea4eeda4d0f8f7d098d485: Log is configured to *not* fsync() on all Append() calls
I20260923 23:14:39.889410 30034 tablet_bootstrap.cc:492] T 4eadff6c00ec4c4aadab2148d0e99be6 P f692378723c742f0a9c65df33d201a6d: No bootstrap required, opened a new log
I20260923 23:14:39.889465 30034 ts_tablet_manager.cc:1397] T 4eadff6c00ec4c4aadab2148d0e99be6 P f692378723c742f0a9c65df33d201a6d: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260923 23:14:39.889575 30035 tablet_bootstrap.cc:492] T 4eadff6c00ec4c4aadab2148d0e99be6 P ffafeada02ea4eeda4d0f8f7d098d485: No bootstrap required, opened a new log
I20260923 23:14:39.889616 30035 ts_tablet_manager.cc:1397] T 4eadff6c00ec4c4aadab2148d0e99be6 P ffafeada02ea4eeda4d0f8f7d098d485: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 23:14:39.891398 30035 raft_consensus.cc:348] T 4eadff6c00ec4c4aadab2148d0e99be6 P ffafeada02ea4eeda4d0f8f7d098d485 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c3e4de1504446d291f74acb1a2b7e1a" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 37057 } } peers { permanent_uuid: "ffafeada02ea4eeda4d0f8f7d098d485" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 35193 } } peers { permanent_uuid: "f692378723c742f0a9c65df33d201a6d" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 38337 } }
I20260923 23:14:39.891510 30035 raft_consensus.cc:374] T 4eadff6c00ec4c4aadab2148d0e99be6 P ffafeada02ea4eeda4d0f8f7d098d485 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:14:39.891530 30035 raft_consensus.cc:729] T 4eadff6c00ec4c4aadab2148d0e99be6 P ffafeada02ea4eeda4d0f8f7d098d485 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffafeada02ea4eeda4d0f8f7d098d485, State: Initialized, Role: FOLLOWER
I20260923 23:14:39.891441 30034 raft_consensus.cc:348] T 4eadff6c00ec4c4aadab2148d0e99be6 P f692378723c742f0a9c65df33d201a6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c3e4de1504446d291f74acb1a2b7e1a" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 37057 } } peers { permanent_uuid: "ffafeada02ea4eeda4d0f8f7d098d485" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 35193 } } peers { permanent_uuid: "f692378723c742f0a9c65df33d201a6d" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 38337 } }
I20260923 23:14:39.891603 30034 raft_consensus.cc:374] T 4eadff6c00ec4c4aadab2148d0e99be6 P f692378723c742f0a9c65df33d201a6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:14:39.891635 30034 raft_consensus.cc:729] T 4eadff6c00ec4c4aadab2148d0e99be6 P f692378723c742f0a9c65df33d201a6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f692378723c742f0a9c65df33d201a6d, State: Initialized, Role: FOLLOWER
I20260923 23:14:39.891686 30035 consensus_queue.cc:260] T 4eadff6c00ec4c4aadab2148d0e99be6 P ffafeada02ea4eeda4d0f8f7d098d485 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c3e4de1504446d291f74acb1a2b7e1a" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 37057 } } peers { permanent_uuid: "ffafeada02ea4eeda4d0f8f7d098d485" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 35193 } } peers { permanent_uuid: "f692378723c742f0a9c65df33d201a6d" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 38337 } }
I20260923 23:14:39.891746 30034 consensus_queue.cc:260] T 4eadff6c00ec4c4aadab2148d0e99be6 P f692378723c742f0a9c65df33d201a6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c3e4de1504446d291f74acb1a2b7e1a" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 37057 } } peers { permanent_uuid: "ffafeada02ea4eeda4d0f8f7d098d485" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 35193 } } peers { permanent_uuid: "f692378723c742f0a9c65df33d201a6d" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 38337 } }
I20260923 23:14:39.891983 30035 ts_tablet_manager.cc:1428] T 4eadff6c00ec4c4aadab2148d0e99be6 P ffafeada02ea4eeda4d0f8f7d098d485: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260923 23:14:39.892035 30034 ts_tablet_manager.cc:1428] T 4eadff6c00ec4c4aadab2148d0e99be6 P f692378723c742f0a9c65df33d201a6d: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260923 23:14:39.892094 29887 heartbeater.cc:503] Master 127.24.227.254:44725 was elected leader, sending a full tablet report...
I20260923 23:14:39.892096 30038 tablet_bootstrap.cc:492] T 4eadff6c00ec4c4aadab2148d0e99be6 P 9c3e4de1504446d291f74acb1a2b7e1a: Bootstrap starting.
I20260923 23:14:39.892131 30020 heartbeater.cc:503] Master 127.24.227.254:44725 was elected leader, sending a full tablet report...
I20260923 23:14:39.892738 30038 tablet_bootstrap.cc:654] T 4eadff6c00ec4c4aadab2148d0e99be6 P 9c3e4de1504446d291f74acb1a2b7e1a: Neither blocks nor log segments found. Creating new log.
I20260923 23:14:39.893362 30038 log.cc:824] T 4eadff6c00ec4c4aadab2148d0e99be6 P 9c3e4de1504446d291f74acb1a2b7e1a: Log is configured to *not* fsync() on all Append() calls
I20260923 23:14:39.894013 30038 tablet_bootstrap.cc:492] T 4eadff6c00ec4c4aadab2148d0e99be6 P 9c3e4de1504446d291f74acb1a2b7e1a: No bootstrap required, opened a new log
I20260923 23:14:39.894089 30038 ts_tablet_manager.cc:1397] T 4eadff6c00ec4c4aadab2148d0e99be6 P 9c3e4de1504446d291f74acb1a2b7e1a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260923 23:14:39.895361 30038 raft_consensus.cc:348] T 4eadff6c00ec4c4aadab2148d0e99be6 P 9c3e4de1504446d291f74acb1a2b7e1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c3e4de1504446d291f74acb1a2b7e1a" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 37057 } } peers { permanent_uuid: "ffafeada02ea4eeda4d0f8f7d098d485" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 35193 } } peers { permanent_uuid: "f692378723c742f0a9c65df33d201a6d" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 38337 } }
I20260923 23:14:39.895490 30038 raft_consensus.cc:374] T 4eadff6c00ec4c4aadab2148d0e99be6 P 9c3e4de1504446d291f74acb1a2b7e1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:14:39.895530 30038 raft_consensus.cc:729] T 4eadff6c00ec4c4aadab2148d0e99be6 P 9c3e4de1504446d291f74acb1a2b7e1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c3e4de1504446d291f74acb1a2b7e1a, State: Initialized, Role: FOLLOWER
I20260923 23:14:39.895624 30038 consensus_queue.cc:260] T 4eadff6c00ec4c4aadab2148d0e99be6 P 9c3e4de1504446d291f74acb1a2b7e1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c3e4de1504446d291f74acb1a2b7e1a" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 37057 } } peers { permanent_uuid: "ffafeada02ea4eeda4d0f8f7d098d485" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 35193 } } peers { permanent_uuid: "f692378723c742f0a9c65df33d201a6d" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 38337 } }
I20260923 23:14:39.895905 29754 heartbeater.cc:503] Master 127.24.227.254:44725 was elected leader, sending a full tablet report...
I20260923 23:14:39.895910 30038 ts_tablet_manager.cc:1428] T 4eadff6c00ec4c4aadab2148d0e99be6 P 9c3e4de1504446d291f74acb1a2b7e1a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260923 23:14:39.903000 30040 raft_consensus.cc:482] T 4eadff6c00ec4c4aadab2148d0e99be6 P f692378723c742f0a9c65df33d201a6d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 23:14:39.903065 30040 raft_consensus.cc:504] T 4eadff6c00ec4c4aadab2148d0e99be6 P f692378723c742f0a9c65df33d201a6d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c3e4de1504446d291f74acb1a2b7e1a" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 37057 } } peers { permanent_uuid: "ffafeada02ea4eeda4d0f8f7d098d485" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 35193 } } peers { permanent_uuid: "f692378723c742f0a9c65df33d201a6d" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 38337 } }
I20260923 23:14:39.903362 30040 leader_election.cc:288] T 4eadff6c00ec4c4aadab2148d0e99be6 P f692378723c742f0a9c65df33d201a6d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c3e4de1504446d291f74acb1a2b7e1a (127.24.227.193:37057), ffafeada02ea4eeda4d0f8f7d098d485 (127.24.227.194:35193)
I20260923 23:14:39.906033 30033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:39.903586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.195:58225 (local address 127.24.227.193:37057)
0923 23:14:39.903624 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:39.903627 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:39.903636 (+ 9us) server_negotiation.cc:407] Connection header received
0923 23:14:39.903673 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:39.903674 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:39.903683 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:39.903733 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:14:39.903916 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:39.904360 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:39.905314 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:39.905690 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:39.905783 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:39.905792 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:39.905946 (+ 154us) server_negotiation.cc:299] Negotiation successful
0923 23:14:39.905978 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260923 23:14:39.906038 30022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:39.903510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.193:37057 (local address 127.24.227.195:58225)
0923 23:14:39.903565 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:39.903575 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:39.903625 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:39.903736 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:39.903737 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:39.903739 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:14:39.903799 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:39.903800 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:39.904469 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:39.904470 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:39.905226 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:39.905228 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:39.905784 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:39.905785 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:39.905857 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:39.905860 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 23:14:39.905938 (+ 78us) client_negotiation.cc:240] Negotiation successful
0923 23:14:39.905975 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":33}
I20260923 23:14:39.906136 30031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:39.903709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.195:49329 (local address 127.24.227.194:35193)
0923 23:14:39.903751 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:39.903753 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:39.903776 (+ 23us) server_negotiation.cc:407] Connection header received
0923 23:14:39.903834 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:39.903835 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:39.903843 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:39.903922 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:14:39.904057 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:39.904502 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:39.905421 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:39.905788 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:39.905871 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:39.905878 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:39.906034 (+ 156us) server_negotiation.cc:299] Negotiation successful
0923 23:14:39.906067 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260923 23:14:39.906327 29708 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4eadff6c00ec4c4aadab2148d0e99be6" candidate_uuid: "f692378723c742f0a9c65df33d201a6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c3e4de1504446d291f74acb1a2b7e1a" is_pre_election: true
I20260923 23:14:39.906424 29708 raft_consensus.cc:2454] T 4eadff6c00ec4c4aadab2148d0e99be6 P 9c3e4de1504446d291f74acb1a2b7e1a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f692378723c742f0a9c65df33d201a6d in term 0.
I20260923 23:14:39.906502 30043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:39.903510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.194:35193 (local address 127.24.227.195:49329)
0923 23:14:39.903723 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:39.903733 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:39.903788 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:39.903922 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:39.903923 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:39.903924 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:14:39.903990 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:39.903991 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:39.904583 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:39.904584 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:39.905333 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:39.905335 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:39.905871 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:39.905871 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:39.905952 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:39.905963 (+ 11us) client_negotiation.cc:769] Sending connection context
0923 23:14:39.906030 (+ 67us) client_negotiation.cc:240] Negotiation successful
0923 23:14:39.906067 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"mutex_wait_us":68,"thread_start_us":77,"threads_started":1}
I20260923 23:14:39.906637 29908 leader_election.cc:302] T 4eadff6c00ec4c4aadab2148d0e99be6 P f692378723c742f0a9c65df33d201a6d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c3e4de1504446d291f74acb1a2b7e1a, f692378723c742f0a9c65df33d201a6d; no voters:
I20260923 23:14:39.906782 30040 raft_consensus.cc:2781] T 4eadff6c00ec4c4aadab2148d0e99be6 P f692378723c742f0a9c65df33d201a6d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 23:14:39.906847 30040 raft_consensus.cc:482] T 4eadff6c00ec4c4aadab2148d0e99be6 P f692378723c742f0a9c65df33d201a6d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 23:14:39.906884 30040 raft_consensus.cc:3037] T 4eadff6c00ec4c4aadab2148d0e99be6 P f692378723c742f0a9c65df33d201a6d [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:14:39.907017 29841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4eadff6c00ec4c4aadab2148d0e99be6" candidate_uuid: "f692378723c742f0a9c65df33d201a6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffafeada02ea4eeda4d0f8f7d098d485" is_pre_election: true
I20260923 23:14:39.907160 29841 raft_consensus.cc:2454] T 4eadff6c00ec4c4aadab2148d0e99be6 P ffafeada02ea4eeda4d0f8f7d098d485 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f692378723c742f0a9c65df33d201a6d in term 0.
I20260923 23:14:39.907596 30040 raft_consensus.cc:504] T 4eadff6c00ec4c4aadab2148d0e99be6 P f692378723c742f0a9c65df33d201a6d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c3e4de1504446d291f74acb1a2b7e1a" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 37057 } } peers { permanent_uuid: "ffafeada02ea4eeda4d0f8f7d098d485" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 35193 } } peers { permanent_uuid: "f692378723c742f0a9c65df33d201a6d" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 38337 } }
I20260923 23:14:39.907716 30040 leader_election.cc:288] T 4eadff6c00ec4c4aadab2148d0e99be6 P f692378723c742f0a9c65df33d201a6d [CANDIDATE]: Term 1 election: Requested vote from peers 9c3e4de1504446d291f74acb1a2b7e1a (127.24.227.193:37057), ffafeada02ea4eeda4d0f8f7d098d485 (127.24.227.194:35193)
I20260923 23:14:39.907811 29708 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4eadff6c00ec4c4aadab2148d0e99be6" candidate_uuid: "f692378723c742f0a9c65df33d201a6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c3e4de1504446d291f74acb1a2b7e1a"
I20260923 23:14:39.907835 29841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4eadff6c00ec4c4aadab2148d0e99be6" candidate_uuid: "f692378723c742f0a9c65df33d201a6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffafeada02ea4eeda4d0f8f7d098d485"
I20260923 23:14:39.907912 29708 raft_consensus.cc:3037] T 4eadff6c00ec4c4aadab2148d0e99be6 P 9c3e4de1504446d291f74acb1a2b7e1a [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:14:39.907927 29841 raft_consensus.cc:3037] T 4eadff6c00ec4c4aadab2148d0e99be6 P ffafeada02ea4eeda4d0f8f7d098d485 [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:14:39.908556 29708 raft_consensus.cc:2454] T 4eadff6c00ec4c4aadab2148d0e99be6 P 9c3e4de1504446d291f74acb1a2b7e1a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f692378723c742f0a9c65df33d201a6d in term 1.
I20260923 23:14:39.908618 29841 raft_consensus.cc:2454] T 4eadff6c00ec4c4aadab2148d0e99be6 P ffafeada02ea4eeda4d0f8f7d098d485 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f692378723c742f0a9c65df33d201a6d in term 1.
I20260923 23:14:39.908761 29908 leader_election.cc:302] T 4eadff6c00ec4c4aadab2148d0e99be6 P f692378723c742f0a9c65df33d201a6d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c3e4de1504446d291f74acb1a2b7e1a, f692378723c742f0a9c65df33d201a6d; no voters:
I20260923 23:14:39.908865 30040 raft_consensus.cc:2781] T 4eadff6c00ec4c4aadab2148d0e99be6 P f692378723c742f0a9c65df33d201a6d [term 1 FOLLOWER]: Leader election won for term 1
I20260923 23:14:39.908993 30040 raft_consensus.cc:686] T 4eadff6c00ec4c4aadab2148d0e99be6 P f692378723c742f0a9c65df33d201a6d [term 1 LEADER]: Becoming Leader. State: Replica: f692378723c742f0a9c65df33d201a6d, State: Running, Role: LEADER
I20260923 23:14:39.909051 30040 consensus_queue.cc:237] T 4eadff6c00ec4c4aadab2148d0e99be6 P f692378723c742f0a9c65df33d201a6d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c3e4de1504446d291f74acb1a2b7e1a" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 37057 } } peers { permanent_uuid: "ffafeada02ea4eeda4d0f8f7d098d485" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 35193 } } peers { permanent_uuid: "f692378723c742f0a9c65df33d201a6d" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 38337 } }
I20260923 23:14:39.909759 29562 catalog_manager.cc:5495] T 4eadff6c00ec4c4aadab2148d0e99be6 P f692378723c742f0a9c65df33d201a6d reported cstate change: term changed from 0 to 1, leader changed from <none> to f692378723c742f0a9c65df33d201a6d (127.24.227.195). New cstate: current_term: 1 leader_uuid: "f692378723c742f0a9c65df33d201a6d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c3e4de1504446d291f74acb1a2b7e1a" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 37057 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ffafeada02ea4eeda4d0f8f7d098d485" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 35193 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f692378723c742f0a9c65df33d201a6d" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 38337 } health_report { overall_health: HEALTHY } } }
I20260923 23:14:39.919740 30032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:39.917838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52818 (local address 127.24.227.195:38337)
0923 23:14:39.917878 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:39.917880 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:39.917904 (+ 24us) server_negotiation.cc:407] Connection header received
0923 23:14:39.917967 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:39.917968 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:39.917980 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:39.918032 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 23:14:39.918132 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:39.918541 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:39.919165 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:39.919326 (+ 161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:39.919377 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:39.919504 (+ 127us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 23:14:39.919580 (+ 76us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 23:14:39.919646 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:39.919676 (+ 30us) server_negotiation.cc:299] Negotiation successful
0923 23:14:39.919688 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260923 23:14:39.921212 29841 raft_consensus.cc:1261] T 4eadff6c00ec4c4aadab2148d0e99be6 P ffafeada02ea4eeda4d0f8f7d098d485 [term 1 FOLLOWER]: Refusing update from remote peer f692378723c742f0a9c65df33d201a6d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 23:14:39.921212 29708 raft_consensus.cc:1261] T 4eadff6c00ec4c4aadab2148d0e99be6 P 9c3e4de1504446d291f74acb1a2b7e1a [term 1 FOLLOWER]: Refusing update from remote peer f692378723c742f0a9c65df33d201a6d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 23:14:39.921550 30040 consensus_queue.cc:1035] T 4eadff6c00ec4c4aadab2148d0e99be6 P f692378723c742f0a9c65df33d201a6d [LEADER]: Connected to new peer: Peer: permanent_uuid: "ffafeada02ea4eeda4d0f8f7d098d485" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 35193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 23:14:39.921669 30044 consensus_queue.cc:1035] T 4eadff6c00ec4c4aadab2148d0e99be6 P f692378723c742f0a9c65df33d201a6d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c3e4de1504446d291f74acb1a2b7e1a" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 37057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 23:14:39.925879 29563 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37914:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 23:14:39.925980 29563 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37914:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 23:14:39.926676 29563 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ecf0fcb657a74edf957248a52b2dce3f: Sending DeleteTablet for 3 replicas of tablet 4eadff6c00ec4c4aadab2148d0e99be6
I20260923 23:14:39.927006 29821 tablet_service.cc:1514] Processing DeleteTablet for tablet 4eadff6c00ec4c4aadab2148d0e99be6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 23:14:39 UTC) from {username='kudu', principal='kudu/127.24.227.254@KRBTEST.COM'} at 127.0.0.1:37476
I20260923 23:14:39.927058 29688 tablet_service.cc:1514] Processing DeleteTablet for tablet 4eadff6c00ec4c4aadab2148d0e99be6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 23:14:39 UTC) from {username='kudu', principal='kudu/127.24.227.254@KRBTEST.COM'} at 127.0.0.1:39012
I20260923 23:14:39.927299 29954 tablet_service.cc:1514] Processing DeleteTablet for tablet 4eadff6c00ec4c4aadab2148d0e99be6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 23:14:39 UTC) from {username='kudu', principal='kudu/127.24.227.254@KRBTEST.COM'} at 127.0.0.1:52802
I20260923 23:14:39.927376 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 29626
I20260923 23:14:39.927497 30059 tablet_replica.cc:330] T 4eadff6c00ec4c4aadab2148d0e99be6 P f692378723c742f0a9c65df33d201a6d: stopping tablet replica
I20260923 23:14:39.927574 30059 raft_consensus.cc:2229] T 4eadff6c00ec4c4aadab2148d0e99be6 P f692378723c742f0a9c65df33d201a6d [term 1 LEADER]: Raft consensus shutting down.
I20260923 23:14:39.927662 30059 raft_consensus.cc:2258] T 4eadff6c00ec4c4aadab2148d0e99be6 P f692378723c742f0a9c65df33d201a6d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 23:14:39.928571 30059 ts_tablet_manager.cc:1905] T 4eadff6c00ec4c4aadab2148d0e99be6 P f692378723c742f0a9c65df33d201a6d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 23:14:39.930205 30059 ts_tablet_manager.cc:1918] T 4eadff6c00ec4c4aadab2148d0e99be6 P f692378723c742f0a9c65df33d201a6d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 23:14:39.930295 30059 log.cc:1196] T 4eadff6c00ec4c4aadab2148d0e99be6 P f692378723c742f0a9c65df33d201a6d: Deleting WAL directory at /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790205259943518-25487-0/minicluster-data/ts-2/wal/wals/4eadff6c00ec4c4aadab2148d0e99be6
I20260923 23:14:39.930534 30059 ts_tablet_manager.cc:1939] T 4eadff6c00ec4c4aadab2148d0e99be6 P f692378723c742f0a9c65df33d201a6d: Deleting consensus metadata
I20260923 23:14:39.930876 29551 catalog_manager.cc:4841] TS f692378723c742f0a9c65df33d201a6d (127.24.227.195:38337): tablet 4eadff6c00ec4c4aadab2148d0e99be6 (table test-table [id=0c2c5152cd694301af451d0badce0277]) successfully deleted
I20260923 23:14:39.931985 30057 tablet_replica.cc:330] T 4eadff6c00ec4c4aadab2148d0e99be6 P ffafeada02ea4eeda4d0f8f7d098d485: stopping tablet replica
I20260923 23:14:39.932085 30057 raft_consensus.cc:2229] T 4eadff6c00ec4c4aadab2148d0e99be6 P ffafeada02ea4eeda4d0f8f7d098d485 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 23:14:39.932174 30057 raft_consensus.cc:2258] T 4eadff6c00ec4c4aadab2148d0e99be6 P ffafeada02ea4eeda4d0f8f7d098d485 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 23:14:39.932504 30057 ts_tablet_manager.cc:1905] T 4eadff6c00ec4c4aadab2148d0e99be6 P ffafeada02ea4eeda4d0f8f7d098d485: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 23:14:39.933698 30057 ts_tablet_manager.cc:1918] T 4eadff6c00ec4c4aadab2148d0e99be6 P ffafeada02ea4eeda4d0f8f7d098d485: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 23:14:39.933772 30057 log.cc:1196] T 4eadff6c00ec4c4aadab2148d0e99be6 P ffafeada02ea4eeda4d0f8f7d098d485: Deleting WAL directory at /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790205259943518-25487-0/minicluster-data/ts-1/wal/wals/4eadff6c00ec4c4aadab2148d0e99be6
I20260923 23:14:39.934065 30057 ts_tablet_manager.cc:1939] T 4eadff6c00ec4c4aadab2148d0e99be6 P ffafeada02ea4eeda4d0f8f7d098d485: Deleting consensus metadata
W20260923 23:14:39.934056 29549 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.227.193:37057 (error 108)
I20260923 23:14:39.934278 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 29759
I20260923 23:14:39.934535 30028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:39.934412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.193:37057 (local address 127.0.0.1:39018)
0923 23:14:39.934478 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:39.934506 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.193:37057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260923 23:14:39.934852 29548 catalog_manager.cc:4841] TS ffafeada02ea4eeda4d0f8f7d098d485 (127.24.227.194:35193): tablet 4eadff6c00ec4c4aadab2148d0e99be6 (table test-table [id=0c2c5152cd694301af451d0badce0277]) successfully deleted
W20260923 23:14:39.936399 29549 catalog_manager.cc:4584] TS 9c3e4de1504446d291f74acb1a2b7e1a (127.24.227.193:37057): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4eadff6c00ec4c4aadab2148d0e99be6: Network error: Client connection negotiation failed: client connection to 127.24.227.193:37057: connect: Connection refused (error 111)
I20260923 23:14:39.939011 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 29892
I20260923 23:14:39.942931 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 29533
2026-09-23T23:14:39Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (1769 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-09-23T23:14:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T23:14:39Z Disabled control of system clock
I20260923 23:14:39.968292 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790205259943518-25487-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790205259943518-25487-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790205259943518-25487-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.254:41089
--webserver_interface=127.24.227.254
--webserver_port=0
--builtin_ntp_servers=127.24.227.212:33595
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.227.254:41089
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260923 23:14:40.044776 30064 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:14:40.044971 30064 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:14:40.044991 30064 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:14:40.046048 30064 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 23:14:40.046080 30064 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:14:40.046093 30064 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:14:40.046105 30064 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 23:14:40.046118 30064 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 23:14:40.047399 30064 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:33595
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790205259943518-25487-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790205259943518-25487-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.227.254:41089
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.254:41089
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.227.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30064
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790205259943518-25487-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:14:40.047580 30064 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:14:40.047801 30064 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:14:40.052402 30070 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:40.052415 30069 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:14:40.052448 30064 server_base.cc:1034] running on GCE node
W20260923 23:14:40.052397 30072 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:14:40.052853 30064 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:14:40.053145 30064 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:14:40.054308 30064 hybrid_clock.cc:648] HybridClock initialized: now 1790205280054302 us; error 36 us; skew 500 ppm
I20260923 23:14:40.055562 30064 webserver.cc:460] Webserver started at http://127.24.227.254:33579/ using document root <none> and password file <none>
I20260923 23:14:40.055784 30064 fs_manager.cc:359] Metadata directory not provided
I20260923 23:14:40.055852 30064 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:14:40.056092 30064 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:14:40.057120 30064 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790205259943518-25487-0/minicluster-data/master-0/data/instance:
uuid: "6234ebf4e249434f8aaef32b1c905890"
format_stamp: "Formatted at 2026-09-23 23:14:40 on dist-test-slave-754n"
server_key: "1cee74f5186a21f05d4ac27b54f7f32e"
server_key_iv: "a5ad69d9fcc3e93143df2aaf3c89a445"
server_key_version: "encryptionkey@0"
I20260923 23:14:40.057466 30064 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790205259943518-25487-0/minicluster-data/master-0/wal/instance:
uuid: "6234ebf4e249434f8aaef32b1c905890"
format_stamp: "Formatted at 2026-09-23 23:14:40 on dist-test-slave-754n"
server_key: "1cee74f5186a21f05d4ac27b54f7f32e"
server_key_iv: "a5ad69d9fcc3e93143df2aaf3c89a445"
server_key_version: "encryptionkey@0"
I20260923 23:14:40.058681 30064 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 23:14:40.059376 30078 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:40.059540 30064 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 23:14:40.059640 30064 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790205259943518-25487-0/minicluster-data/master-0/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790205259943518-25487-0/minicluster-data/master-0/wal
uuid: "6234ebf4e249434f8aaef32b1c905890"
format_stamp: "Formatted at 2026-09-23 23:14:40 on dist-test-slave-754n"
server_key: "1cee74f5186a21f05d4ac27b54f7f32e"
server_key_iv: "a5ad69d9fcc3e93143df2aaf3c89a445"
server_key_version: "encryptionkey@0"
I20260923 23:14:40.059733 30064 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790205259943518-25487-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790205259943518-25487-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790205259943518-25487-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:14:40.063026 30064 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:14:40.063275 30064 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:14:40.063411 30064 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:14:40.067128 30064 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.254:41089
I20260923 23:14:40.067154 30130 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.254:41089 every 8 connection(s)
I20260923 23:14:40.067466 30064 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
I20260923 23:14:40.068203 30131 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:14:40.070767 30131 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6234ebf4e249434f8aaef32b1c905890: Bootstrap starting.
I20260923 23:14:40.071306 30131 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6234ebf4e249434f8aaef32b1c905890: Neither blocks nor log segments found. Creating new log.
I20260923 23:14:40.071596 30131 log.cc:824] T 00000000000000000000000000000000 P 6234ebf4e249434f8aaef32b1c905890: Log is configured to *not* fsync() on all Append() calls
I20260923 23:14:40.071673 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 30064
I20260923 23:14:40.071785 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790205259943518-25487-0/minicluster-data/master-0/wal/instance
I20260923 23:14:40.071995 25487 external_mini_cluster.cc:1428] Setting key 36c45edf32400bda7760e8517eddd904
I20260923 23:14:40.072409 30131 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6234ebf4e249434f8aaef32b1c905890: No bootstrap required, opened a new log
I20260923 23:14:40.074509 30131 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6234ebf4e249434f8aaef32b1c905890 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6234ebf4e249434f8aaef32b1c905890" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41089 } }
I20260923 23:14:40.074677 30131 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6234ebf4e249434f8aaef32b1c905890 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:14:40.074718 30131 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6234ebf4e249434f8aaef32b1c905890 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6234ebf4e249434f8aaef32b1c905890, State: Initialized, Role: FOLLOWER
I20260923 23:14:40.074855 30135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:40.072477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48776 (local address 127.24.227.254:41089)
0923 23:14:40.072673 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:40.072681 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:40.072720 (+ 39us) server_negotiation.cc:407] Connection header received
0923 23:14:40.073607 (+ 887us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:40.073615 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:40.073664 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:40.073810 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:40.074437 (+ 627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:40.074441 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:40.074441 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 23:14:40.074446 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:40.074490 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 23:14:40.074505 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:40.074582 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:40.074634 (+ 52us) server_negotiation.cc:299] Negotiation successful
0923 23:14:40.074688 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":77,"threads_started":1}
I20260923 23:14:40.074829 30131 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6234ebf4e249434f8aaef32b1c905890 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6234ebf4e249434f8aaef32b1c905890" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41089 } }
I20260923 23:14:40.074919 30131 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6234ebf4e249434f8aaef32b1c905890 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 23:14:40.074954 30131 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6234ebf4e249434f8aaef32b1c905890 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 23:14:40.074998 30131 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6234ebf4e249434f8aaef32b1c905890 [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:14:40.075693 30131 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6234ebf4e249434f8aaef32b1c905890 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6234ebf4e249434f8aaef32b1c905890" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41089 } }
I20260923 23:14:40.075793 30131 leader_election.cc:302] T 00000000000000000000000000000000 P 6234ebf4e249434f8aaef32b1c905890 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6234ebf4e249434f8aaef32b1c905890; no voters:
I20260923 23:14:40.075960 30131 leader_election.cc:288] T 00000000000000000000000000000000 P 6234ebf4e249434f8aaef32b1c905890 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 23:14:40.075985 30136 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6234ebf4e249434f8aaef32b1c905890 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 23:14:40.076084 30136 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6234ebf4e249434f8aaef32b1c905890 [term 1 LEADER]: Becoming Leader. State: Replica: 6234ebf4e249434f8aaef32b1c905890, State: Running, Role: LEADER
I20260923 23:14:40.076164 30136 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6234ebf4e249434f8aaef32b1c905890 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6234ebf4e249434f8aaef32b1c905890" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41089 } }
I20260923 23:14:40.076205 30131 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6234ebf4e249434f8aaef32b1c905890 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 23:14:40.076570 30137 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6234ebf4e249434f8aaef32b1c905890 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6234ebf4e249434f8aaef32b1c905890" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6234ebf4e249434f8aaef32b1c905890" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41089 } } }
I20260923 23:14:40.076601 30138 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6234ebf4e249434f8aaef32b1c905890 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6234ebf4e249434f8aaef32b1c905890. Latest consensus state: current_term: 1 leader_uuid: "6234ebf4e249434f8aaef32b1c905890" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6234ebf4e249434f8aaef32b1c905890" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41089 } } }
I20260923 23:14:40.076758 30138 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6234ebf4e249434f8aaef32b1c905890 [sys.catalog]: This master's current role is: LEADER
I20260923 23:14:40.076963 30145 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 23:14:40.077142 30137 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6234ebf4e249434f8aaef32b1c905890 [sys.catalog]: This master's current role is: LEADER
I20260923 23:14:40.077471 30145 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 23:14:40.078917 30145 catalog_manager.cc:1347] Generated new cluster ID: 953a36496ceb44b7a5af59333d97de1a
I20260923 23:14:40.078965 30145 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 23:14:40.099841 30145 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 23:14:40.100019 30145 catalog_manager.cc:1504] Loading token signing keys...
I20260923 23:14:40.107064 30145 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6234ebf4e249434f8aaef32b1c905890: Generated new TSK 0
I20260923 23:14:40.107225 30145 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 23:14:40.110306 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790205259943518-25487-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790205259943518-25487-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790205259943518-25487-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790205259943518-25487-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.227.193:0
--local_ip_for_outbound_sockets=127.24.227.193
--webserver_interface=127.24.227.193
--webserver_port=0
--tserver_master_addrs=127.24.227.254:41089
--builtin_ntp_servers=127.24.227.212:33595
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260923 23:14:40.188488 30155 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:14:40.188699 30155 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:14:40.188735 30155 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:14:40.189880 30155 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:14:40.189946 30155 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:14:40.189991 30155 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.227.193
I20260923 23:14:40.191295 30155 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:33595
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790205259943518-25487-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790205259943518-25487-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.227.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790205259943518-25487-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.227.193
--webserver_port=0
--tserver_master_addrs=127.24.227.254:41089
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30155
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.227.193
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790205259943518-25487-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:14:40.191553 30155 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:14:40.191821 30155 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260923 23:14:40.197121 30155 server_base.cc:1034] running on GCE node
W20260923 23:14:40.197052 30161 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:40.197063 30163 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:40.197080 30160 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:14:40.197504 30155 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:14:40.197700 30155 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:14:40.198849 30155 hybrid_clock.cc:648] HybridClock initialized: now 1790205280198811 us; error 37 us; skew 500 ppm
I20260923 23:14:40.200085 30155 webserver.cc:460] Webserver started at http://127.24.227.193:35139/ using document root <none> and password file <none>
I20260923 23:14:40.200291 30155 fs_manager.cc:359] Metadata directory not provided
I20260923 23:14:40.200332 30155 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:14:40.200464 30155 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:14:40.201473 30155 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790205259943518-25487-0/minicluster-data/ts-0/data/instance:
uuid: "951c99ff621647d5bd03cc2439ad069f"
format_stamp: "Formatted at 2026-09-23 23:14:40 on dist-test-slave-754n"
server_key: "c5e72b94a62f551a34ab90094b5f0256"
server_key_iv: "b1de2106d645e941c65d89cb48c2cf79"
server_key_version: "encryptionkey@0"
I20260923 23:14:40.201820 30155 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790205259943518-25487-0/minicluster-data/ts-0/wal/instance:
uuid: "951c99ff621647d5bd03cc2439ad069f"
format_stamp: "Formatted at 2026-09-23 23:14:40 on dist-test-slave-754n"
server_key: "c5e72b94a62f551a34ab90094b5f0256"
server_key_iv: "b1de2106d645e941c65d89cb48c2cf79"
server_key_version: "encryptionkey@0"
I20260923 23:14:40.203033 30155 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 23:14:40.203672 30169 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:40.203828 30155 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 23:14:40.203958 30155 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790205259943518-25487-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790205259943518-25487-0/minicluster-data/ts-0/wal
uuid: "951c99ff621647d5bd03cc2439ad069f"
format_stamp: "Formatted at 2026-09-23 23:14:40 on dist-test-slave-754n"
server_key: "c5e72b94a62f551a34ab90094b5f0256"
server_key_iv: "b1de2106d645e941c65d89cb48c2cf79"
server_key_version: "encryptionkey@0"
I20260923 23:14:40.204031 30155 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790205259943518-25487-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790205259943518-25487-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790205259943518-25487-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:14:40.207191 30155 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:14:40.207422 30155 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:14:40.207558 30155 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:14:40.207756 30155 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 23:14:40.208068 30155 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 23:14:40.208119 30155 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:40.208158 30155 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 23:14:40.208184 30155 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:40.213498 30155 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.193:44505
I20260923 23:14:40.213536 30282 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.193:44505 every 8 connection(s)
I20260923 23:14:40.213826 30155 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790205259943518-25487-0/minicluster-data/ts-0/data/info.pb
I20260923 23:14:40.214787 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 30155
I20260923 23:14:40.214890 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790205259943518-25487-0/minicluster-data/ts-0/wal/instance
I20260923 23:14:40.215067 25487 external_mini_cluster.cc:1428] Setting key efcd01be8c057f301e81ba236175287c
I20260923 23:14:40.216006 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790205259943518-25487-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790205259943518-25487-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790205259943518-25487-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790205259943518-25487-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.227.194:0
--local_ip_for_outbound_sockets=127.24.227.194
--webserver_interface=127.24.227.194
--webserver_port=0
--tserver_master_addrs=127.24.227.254:41089
--builtin_ntp_servers=127.24.227.212:33595
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260923 23:14:40.216835 30135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:40.214491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.193:39729 (local address 127.24.227.254:41089)
0923 23:14:40.214546 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:40.214548 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:40.214732 (+ 184us) server_negotiation.cc:407] Connection header received
0923 23:14:40.215328 (+ 596us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:40.215329 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:40.215335 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:40.215392 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:40.216587 (+ 1195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:40.216588 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:40.216589 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 23:14:40.216593 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:40.216611 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 23:14:40.216615 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:40.216684 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:40.216728 (+ 44us) server_negotiation.cc:299] Negotiation successful
0923 23:14:40.216741 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260923 23:14:40.216953 30285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:40.214453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41089 (local address 127.24.227.193:39729)
0923 23:14:40.214611 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:40.214648 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:40.215221 (+ 573us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:40.215452 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:40.215456 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:40.215465 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:14:40.216451 (+ 986us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 23:14:40.216451 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:14:40.216476 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 23:14:40.216501 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:14:40.216681 (+ 180us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:14:40.216683 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:14:40.216685 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 23:14:40.216732 (+ 47us) client_negotiation.cc:240] Negotiation successful
0923 23:14:40.216772 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":79,"threads_started":1}
I20260923 23:14:40.217429 30283 heartbeater.cc:347] Connected to a master server at 127.24.227.254:41089
I20260923 23:14:40.217527 30283 heartbeater.cc:464] Registering TS with master...
I20260923 23:14:40.217655 30283 heartbeater.cc:511] Master 127.24.227.254:41089 requested a full tablet report, sending...
I20260923 23:14:40.218108 30095 ts_manager.cc:194] Registered new tserver with Master: 951c99ff621647d5bd03cc2439ad069f (127.24.227.193:44505)
W20260923 23:14:40.292894 30286 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:14:40.293074 30286 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:14:40.293095 30286 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:14:40.294224 30286 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:14:40.294261 30286 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:14:40.294313 30286 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.227.194
I20260923 23:14:40.295589 30286 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:33595
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790205259943518-25487-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790205259943518-25487-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.227.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790205259943518-25487-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.227.194
--webserver_port=0
--tserver_master_addrs=127.24.227.254:41089
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30286
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.227.194
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790205259943518-25487-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:14:40.295812 30286 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:14:40.296128 30286 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:14:40.300649 30291 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:40.300671 30294 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:40.300675 30292 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:14:40.300891 30286 server_base.cc:1034] running on GCE node
I20260923 23:14:40.301045 30286 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:14:40.301226 30286 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:14:40.302392 30286 hybrid_clock.cc:648] HybridClock initialized: now 1790205280302370 us; error 35 us; skew 500 ppm
I20260923 23:14:40.303572 30286 webserver.cc:460] Webserver started at http://127.24.227.194:45171/ using document root <none> and password file <none>
I20260923 23:14:40.303789 30286 fs_manager.cc:359] Metadata directory not provided
I20260923 23:14:40.303854 30286 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:14:40.304032 30286 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:14:40.305024 30286 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790205259943518-25487-0/minicluster-data/ts-1/data/instance:
uuid: "abb560de64b04b9298c53a850739722d"
format_stamp: "Formatted at 2026-09-23 23:14:40 on dist-test-slave-754n"
server_key: "d8cef0d703923a28923384fe0ff7f845"
server_key_iv: "340b152f458f97abe59dacabb0d812ac"
server_key_version: "encryptionkey@0"
I20260923 23:14:40.305392 30286 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790205259943518-25487-0/minicluster-data/ts-1/wal/instance:
uuid: "abb560de64b04b9298c53a850739722d"
format_stamp: "Formatted at 2026-09-23 23:14:40 on dist-test-slave-754n"
server_key: "d8cef0d703923a28923384fe0ff7f845"
server_key_iv: "340b152f458f97abe59dacabb0d812ac"
server_key_version: "encryptionkey@0"
I20260923 23:14:40.306773 30286 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 23:14:40.307603 30300 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:40.307761 30286 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 23:14:40.307866 30286 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790205259943518-25487-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790205259943518-25487-0/minicluster-data/ts-1/wal
uuid: "abb560de64b04b9298c53a850739722d"
format_stamp: "Formatted at 2026-09-23 23:14:40 on dist-test-slave-754n"
server_key: "d8cef0d703923a28923384fe0ff7f845"
server_key_iv: "340b152f458f97abe59dacabb0d812ac"
server_key_version: "encryptionkey@0"
I20260923 23:14:40.307963 30286 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790205259943518-25487-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790205259943518-25487-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790205259943518-25487-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:14:40.311088 30286 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:14:40.311340 30286 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:14:40.311493 30286 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:14:40.311689 30286 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 23:14:40.312027 30286 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 23:14:40.312078 30286 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:40.312116 30286 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 23:14:40.312142 30286 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:40.317682 30286 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.194:39797
I20260923 23:14:40.317711 30413 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.194:39797 every 8 connection(s)
I20260923 23:14:40.317979 30286 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790205259943518-25487-0/minicluster-data/ts-1/data/info.pb
I20260923 23:14:40.320585 30135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:40.318558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.194:58951 (local address 127.24.227.254:41089)
0923 23:14:40.318628 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:40.318631 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:40.318802 (+ 171us) server_negotiation.cc:407] Connection header received
0923 23:14:40.319213 (+ 411us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:40.319214 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:40.319219 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:40.319268 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:40.320330 (+ 1062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:40.320331 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:40.320331 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 23:14:40.320335 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:40.320346 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 23:14:40.320349 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:40.320410 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:40.320510 (+ 100us) server_negotiation.cc:299] Negotiation successful
0923 23:14:40.320521 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260923 23:14:40.320597 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 30286
I20260923 23:14:40.320698 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790205259943518-25487-0/minicluster-data/ts-1/wal/instance
I20260923 23:14:40.320876 25487 external_mini_cluster.cc:1428] Setting key f2e4dafd29b81002b819aed425ddd26f
I20260923 23:14:40.320885 30416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:40.318536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41089 (local address 127.24.227.194:58951)
0923 23:14:40.318703 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:40.318729 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:40.319132 (+ 403us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:40.319319 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:40.319322 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:40.319329 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:14:40.320203 (+ 874us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 23:14:40.320203 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:14:40.320227 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 23:14:40.320252 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:14:40.320429 (+ 177us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:14:40.320433 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:14:40.320436 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 23:14:40.320516 (+ 80us) client_negotiation.cc:240] Negotiation successful
0923 23:14:40.320594 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":93,"threads_started":1}
I20260923 23:14:40.321345 30414 heartbeater.cc:347] Connected to a master server at 127.24.227.254:41089
I20260923 23:14:40.321465 30414 heartbeater.cc:464] Registering TS with master...
I20260923 23:14:40.321601 30414 heartbeater.cc:511] Master 127.24.227.254:41089 requested a full tablet report, sending...
I20260923 23:14:40.321794 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790205259943518-25487-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790205259943518-25487-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790205259943518-25487-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790205259943518-25487-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.227.195:0
--local_ip_for_outbound_sockets=127.24.227.195
--webserver_interface=127.24.227.195
--webserver_port=0
--tserver_master_addrs=127.24.227.254:41089
--builtin_ntp_servers=127.24.227.212:33595
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260923 23:14:40.322014 30095 ts_manager.cc:194] Registered new tserver with Master: abb560de64b04b9298c53a850739722d (127.24.227.194:39797)
W20260923 23:14:40.396632 30417 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:14:40.396796 30417 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:14:40.396816 30417 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:14:40.397890 30417 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:14:40.397925 30417 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:14:40.397950 30417 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.227.195
I20260923 23:14:40.399196 30417 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:33595
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790205259943518-25487-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790205259943518-25487-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.227.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790205259943518-25487-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.227.195
--webserver_port=0
--tserver_master_addrs=127.24.227.254:41089
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30417
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.227.195
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790205259943518-25487-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:14:40.399348 30417 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:14:40.399603 30417 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:14:40.404145 30422 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:40.404163 30425 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:40.404145 30423 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:14:40.404366 30417 server_base.cc:1034] running on GCE node
I20260923 23:14:40.404532 30417 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:14:40.404692 30417 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:14:40.405866 30417 hybrid_clock.cc:648] HybridClock initialized: now 1790205280405838 us; error 40 us; skew 500 ppm
I20260923 23:14:40.407027 30417 webserver.cc:460] Webserver started at http://127.24.227.195:46519/ using document root <none> and password file <none>
I20260923 23:14:40.407265 30417 fs_manager.cc:359] Metadata directory not provided
I20260923 23:14:40.407308 30417 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:14:40.407450 30417 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:14:40.408511 30417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790205259943518-25487-0/minicluster-data/ts-2/data/instance:
uuid: "6cb2976444114a38bdd7891bea5f0c31"
format_stamp: "Formatted at 2026-09-23 23:14:40 on dist-test-slave-754n"
server_key: "9583496670eb765527a2ddf89a6c6455"
server_key_iv: "09ad3ab9f3f5973d9d76dbdd74906f06"
server_key_version: "encryptionkey@0"
I20260923 23:14:40.408820 30417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790205259943518-25487-0/minicluster-data/ts-2/wal/instance:
uuid: "6cb2976444114a38bdd7891bea5f0c31"
format_stamp: "Formatted at 2026-09-23 23:14:40 on dist-test-slave-754n"
server_key: "9583496670eb765527a2ddf89a6c6455"
server_key_iv: "09ad3ab9f3f5973d9d76dbdd74906f06"
server_key_version: "encryptionkey@0"
I20260923 23:14:40.410008 30417 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 23:14:40.410727 30431 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:40.410889 30417 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 23:14:40.410996 30417 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790205259943518-25487-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790205259943518-25487-0/minicluster-data/ts-2/wal
uuid: "6cb2976444114a38bdd7891bea5f0c31"
format_stamp: "Formatted at 2026-09-23 23:14:40 on dist-test-slave-754n"
server_key: "9583496670eb765527a2ddf89a6c6455"
server_key_iv: "09ad3ab9f3f5973d9d76dbdd74906f06"
server_key_version: "encryptionkey@0"
I20260923 23:14:40.411069 30417 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790205259943518-25487-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790205259943518-25487-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790205259943518-25487-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:14:40.414238 30417 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:14:40.414486 30417 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:14:40.414623 30417 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:14:40.414814 30417 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 23:14:40.415091 30417 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 23:14:40.415139 30417 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:40.415174 30417 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 23:14:40.415199 30417 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:40.420325 30417 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.195:42129
I20260923 23:14:40.420382 30544 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.195:42129 every 8 connection(s)
I20260923 23:14:40.420662 30417 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790205259943518-25487-0/minicluster-data/ts-2/data/info.pb
I20260923 23:14:40.423086 30135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:40.421258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.195:41319 (local address 127.24.227.254:41089)
0923 23:14:40.421330 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:40.421332 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:40.421501 (+ 169us) server_negotiation.cc:407] Connection header received
0923 23:14:40.421865 (+ 364us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:40.421866 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:40.421870 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:40.421921 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:40.422893 (+ 972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:40.422894 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:40.422894 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 23:14:40.422897 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:40.422909 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 23:14:40.422913 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:40.422962 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:40.423024 (+ 62us) server_negotiation.cc:299] Negotiation successful
0923 23:14:40.423035 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260923 23:14:40.423266 30547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:40.421213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41089 (local address 127.24.227.195:41319)
0923 23:14:40.421371 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:40.421398 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:40.421788 (+ 390us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:40.421969 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:40.421972 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:40.421979 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:14:40.422754 (+ 775us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 23:14:40.422754 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:14:40.422776 (+ 22us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 23:14:40.422803 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:14:40.422966 (+ 163us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:14:40.422969 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:14:40.422971 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 23:14:40.423018 (+ 47us) client_negotiation.cc:240] Negotiation successful
0923 23:14:40.423059 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":81,"threads_started":1}
I20260923 23:14:40.423656 30545 heartbeater.cc:347] Connected to a master server at 127.24.227.254:41089
I20260923 23:14:40.423739 30545 heartbeater.cc:464] Registering TS with master...
I20260923 23:14:40.423833 30545 heartbeater.cc:511] Master 127.24.227.254:41089 requested a full tablet report, sending...
I20260923 23:14:40.424204 30095 ts_manager.cc:194] Registered new tserver with Master: 6cb2976444114a38bdd7891bea5f0c31 (127.24.227.195:42129)
I20260923 23:14:40.426092 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 30417
I20260923 23:14:40.426189 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790205259943518-25487-0/minicluster-data/ts-2/wal/instance
I20260923 23:14:40.426369 25487 external_mini_cluster.cc:1428] Setting key bfa9634c5ac15c7f0d88f7d2b0464e7f
I20260923 23:14:40.427088 25487 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 23:14:40.429119 30135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:40.428312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48782 (local address 127.24.227.254:41089)
0923 23:14:40.428358 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:40.428360 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:40.428469 (+ 109us) server_negotiation.cc:407] Connection header received
0923 23:14:40.428516 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:40.428516 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:40.428519 (+ 3us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:40.428561 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:40.428971 (+ 410us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:40.428972 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:40.428972 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 23:14:40.428975 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:40.428984 (+ 9us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 23:14:40.428987 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:40.429032 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:40.429064 (+ 32us) server_negotiation.cc:299] Negotiation successful
0923 23:14:40.429073 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260923 23:14:40.430459 30095 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48782:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 23:14:40.431927 30095 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 23:14:40.435496 30556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:40.434191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40964 (local address 127.24.227.195:42129)
0923 23:14:40.434333 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:40.434337 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:40.434344 (+ 7us) server_negotiation.cc:407] Connection header received
0923 23:14:40.434362 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:40.434364 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:40.434370 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:40.434463 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:40.435083 (+ 620us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:40.435089 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:40.435089 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 23:14:40.435094 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:40.435126 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 23:14:40.435143 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:40.435203 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:40.435318 (+ 115us) server_negotiation.cc:299] Negotiation successful
0923 23:14:40.435333 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":71,"threads_started":1}
I20260923 23:14:40.435563 30553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:40.433991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.195:42129 (local address 127.0.0.1:40964)
0923 23:14:40.434174 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:40.434207 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:40.434268 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:40.434474 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:40.434479 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:40.434484 (+ 5us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:14:40.434970 (+ 486us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 23:14:40.434971 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:14:40.434985 (+ 14us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 23:14:40.435008 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:14:40.435200 (+ 192us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:14:40.435205 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:14:40.435208 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 23:14:40.435310 (+ 102us) client_negotiation.cc:240] Negotiation successful
0923 23:14:40.435493 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":75,"threads_started":1}
I20260923 23:14:40.436015 30555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:40.433995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.193:44505 (local address 127.0.0.1:46256)
0923 23:14:40.434254 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:40.434266 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:40.434312 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:40.434977 (+ 665us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:40.434978 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:40.434980 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:14:40.435411 (+ 431us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 23:14:40.435412 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:14:40.435418 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 23:14:40.435430 (+ 12us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:14:40.435886 (+ 456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:14:40.435887 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:14:40.435888 (+ 1us) client_negotiation.cc:769] Sending connection context
0923 23:14:40.435935 (+ 47us) client_negotiation.cc:240] Negotiation successful
0923 23:14:40.435948 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":93,"threads_started":1}
I20260923 23:14:40.436029 30558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:40.434389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46256 (local address 127.24.227.193:44505)
0923 23:14:40.434547 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:40.434551 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:40.434562 (+ 11us) server_negotiation.cc:407] Connection header received
0923 23:14:40.434596 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:40.434598 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:40.434606 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:40.434981 (+ 375us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:40.435753 (+ 772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:40.435757 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:40.435757 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 23:14:40.435760 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:40.435785 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 23:14:40.435798 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:40.435885 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:40.435942 (+ 57us) server_negotiation.cc:299] Negotiation successful
0923 23:14:40.435956 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":94,"threads_started":1}
I20260923 23:14:40.436156 30479 tablet_service.cc:1467] Processing CreateTablet for tablet 5996cbfea86e4ef59ce6ee492d4c989f (DEFAULT_TABLE table=test-table [id=4d88b0492adb4c37a8f6f3b8079d8749]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 23:14:40.436455 30479 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5996cbfea86e4ef59ce6ee492d4c989f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:14:40.436470 30554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:40.433995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.194:39797 (local address 127.0.0.1:59524)
0923 23:14:40.434806 (+ 811us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:40.434815 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:40.434860 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:40.435317 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:40.435318 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:40.435320 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:14:40.435815 (+ 495us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 23:14:40.435816 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:14:40.435821 (+ 5us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 23:14:40.435830 (+ 9us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:14:40.436359 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:14:40.436360 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:14:40.436360 (+ 0us) client_negotiation.cc:769] Sending connection context
0923 23:14:40.436406 (+ 46us) client_negotiation.cc:240] Negotiation successful
0923 23:14:40.436417 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":797,"mutex_wait_us":6,"thread_start_us":53,"threads_started":1}
I20260923 23:14:40.436681 30557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:40.434228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59524 (local address 127.24.227.194:39797)
0923 23:14:40.434756 (+ 528us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:40.434760 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:40.434923 (+ 163us) server_negotiation.cc:407] Connection header received
0923 23:14:40.434953 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:40.434955 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:40.434974 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:40.435050 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:40.436004 (+ 954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:40.436010 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:40.436010 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 23:14:40.436015 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:40.436048 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 23:14:40.436280 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:40.436335 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:40.436421 (+ 86us) server_negotiation.cc:299] Negotiation successful
0923 23:14:40.436437 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":74,"threads_started":1}
I20260923 23:14:40.436653 30217 tablet_service.cc:1467] Processing CreateTablet for tablet 5996cbfea86e4ef59ce6ee492d4c989f (DEFAULT_TABLE table=test-table [id=4d88b0492adb4c37a8f6f3b8079d8749]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 23:14:40.436956 30217 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5996cbfea86e4ef59ce6ee492d4c989f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:14:40.437314 30348 tablet_service.cc:1467] Processing CreateTablet for tablet 5996cbfea86e4ef59ce6ee492d4c989f (DEFAULT_TABLE table=test-table [id=4d88b0492adb4c37a8f6f3b8079d8749]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 23:14:40.437633 30348 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5996cbfea86e4ef59ce6ee492d4c989f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:14:40.439213 30559 tablet_bootstrap.cc:492] T 5996cbfea86e4ef59ce6ee492d4c989f P 951c99ff621647d5bd03cc2439ad069f: Bootstrap starting.
I20260923 23:14:40.439499 30560 tablet_bootstrap.cc:492] T 5996cbfea86e4ef59ce6ee492d4c989f P 6cb2976444114a38bdd7891bea5f0c31: Bootstrap starting.
I20260923 23:14:40.439689 30559 tablet_bootstrap.cc:654] T 5996cbfea86e4ef59ce6ee492d4c989f P 951c99ff621647d5bd03cc2439ad069f: Neither blocks nor log segments found. Creating new log.
I20260923 23:14:40.439949 30559 log.cc:824] T 5996cbfea86e4ef59ce6ee492d4c989f P 951c99ff621647d5bd03cc2439ad069f: Log is configured to *not* fsync() on all Append() calls
I20260923 23:14:40.440119 30560 tablet_bootstrap.cc:654] T 5996cbfea86e4ef59ce6ee492d4c989f P 6cb2976444114a38bdd7891bea5f0c31: Neither blocks nor log segments found. Creating new log.
I20260923 23:14:40.440407 30560 log.cc:824] T 5996cbfea86e4ef59ce6ee492d4c989f P 6cb2976444114a38bdd7891bea5f0c31: Log is configured to *not* fsync() on all Append() calls
I20260923 23:14:40.440670 30559 tablet_bootstrap.cc:492] T 5996cbfea86e4ef59ce6ee492d4c989f P 951c99ff621647d5bd03cc2439ad069f: No bootstrap required, opened a new log
I20260923 23:14:40.440729 30559 ts_tablet_manager.cc:1397] T 5996cbfea86e4ef59ce6ee492d4c989f P 951c99ff621647d5bd03cc2439ad069f: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260923 23:14:40.442648 30559 raft_consensus.cc:348] T 5996cbfea86e4ef59ce6ee492d4c989f P 951c99ff621647d5bd03cc2439ad069f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cb2976444114a38bdd7891bea5f0c31" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 42129 } } peers { permanent_uuid: "951c99ff621647d5bd03cc2439ad069f" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 44505 } } peers { permanent_uuid: "abb560de64b04b9298c53a850739722d" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 39797 } }
I20260923 23:14:40.442795 30559 raft_consensus.cc:374] T 5996cbfea86e4ef59ce6ee492d4c989f P 951c99ff621647d5bd03cc2439ad069f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:14:40.442839 30559 raft_consensus.cc:729] T 5996cbfea86e4ef59ce6ee492d4c989f P 951c99ff621647d5bd03cc2439ad069f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 951c99ff621647d5bd03cc2439ad069f, State: Initialized, Role: FOLLOWER
I20260923 23:14:40.442955 30559 consensus_queue.cc:260] T 5996cbfea86e4ef59ce6ee492d4c989f P 951c99ff621647d5bd03cc2439ad069f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cb2976444114a38bdd7891bea5f0c31" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 42129 } } peers { permanent_uuid: "951c99ff621647d5bd03cc2439ad069f" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 44505 } } peers { permanent_uuid: "abb560de64b04b9298c53a850739722d" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 39797 } }
I20260923 23:14:40.443220 30559 ts_tablet_manager.cc:1428] T 5996cbfea86e4ef59ce6ee492d4c989f P 951c99ff621647d5bd03cc2439ad069f: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260923 23:14:40.443248 30283 heartbeater.cc:503] Master 127.24.227.254:41089 was elected leader, sending a full tablet report...
I20260923 23:14:40.443400 30562 tablet_bootstrap.cc:492] T 5996cbfea86e4ef59ce6ee492d4c989f P abb560de64b04b9298c53a850739722d: Bootstrap starting.
I20260923 23:14:40.444031 30562 tablet_bootstrap.cc:654] T 5996cbfea86e4ef59ce6ee492d4c989f P abb560de64b04b9298c53a850739722d: Neither blocks nor log segments found. Creating new log.
I20260923 23:14:40.444362 30562 log.cc:824] T 5996cbfea86e4ef59ce6ee492d4c989f P abb560de64b04b9298c53a850739722d: Log is configured to *not* fsync() on all Append() calls
I20260923 23:14:40.445020 30562 tablet_bootstrap.cc:492] T 5996cbfea86e4ef59ce6ee492d4c989f P abb560de64b04b9298c53a850739722d: No bootstrap required, opened a new log
I20260923 23:14:40.445032 30560 tablet_bootstrap.cc:492] T 5996cbfea86e4ef59ce6ee492d4c989f P 6cb2976444114a38bdd7891bea5f0c31: No bootstrap required, opened a new log
I20260923 23:14:40.445088 30560 ts_tablet_manager.cc:1397] T 5996cbfea86e4ef59ce6ee492d4c989f P 6cb2976444114a38bdd7891bea5f0c31: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260923 23:14:40.445097 30562 ts_tablet_manager.cc:1397] T 5996cbfea86e4ef59ce6ee492d4c989f P abb560de64b04b9298c53a850739722d: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260923 23:14:40.446836 30560 raft_consensus.cc:348] T 5996cbfea86e4ef59ce6ee492d4c989f P 6cb2976444114a38bdd7891bea5f0c31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cb2976444114a38bdd7891bea5f0c31" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 42129 } } peers { permanent_uuid: "951c99ff621647d5bd03cc2439ad069f" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 44505 } } peers { permanent_uuid: "abb560de64b04b9298c53a850739722d" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 39797 } }
I20260923 23:14:40.446947 30560 raft_consensus.cc:374] T 5996cbfea86e4ef59ce6ee492d4c989f P 6cb2976444114a38bdd7891bea5f0c31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:14:40.446856 30562 raft_consensus.cc:348] T 5996cbfea86e4ef59ce6ee492d4c989f P abb560de64b04b9298c53a850739722d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cb2976444114a38bdd7891bea5f0c31" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 42129 } } peers { permanent_uuid: "951c99ff621647d5bd03cc2439ad069f" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 44505 } } peers { permanent_uuid: "abb560de64b04b9298c53a850739722d" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 39797 } }
I20260923 23:14:40.446971 30560 raft_consensus.cc:729] T 5996cbfea86e4ef59ce6ee492d4c989f P 6cb2976444114a38bdd7891bea5f0c31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cb2976444114a38bdd7891bea5f0c31, State: Initialized, Role: FOLLOWER
I20260923 23:14:40.446990 30562 raft_consensus.cc:374] T 5996cbfea86e4ef59ce6ee492d4c989f P abb560de64b04b9298c53a850739722d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:14:40.447034 30562 raft_consensus.cc:729] T 5996cbfea86e4ef59ce6ee492d4c989f P abb560de64b04b9298c53a850739722d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abb560de64b04b9298c53a850739722d, State: Initialized, Role: FOLLOWER
I20260923 23:14:40.447079 30560 consensus_queue.cc:260] T 5996cbfea86e4ef59ce6ee492d4c989f P 6cb2976444114a38bdd7891bea5f0c31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cb2976444114a38bdd7891bea5f0c31" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 42129 } } peers { permanent_uuid: "951c99ff621647d5bd03cc2439ad069f" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 44505 } } peers { permanent_uuid: "abb560de64b04b9298c53a850739722d" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 39797 } }
I20260923 23:14:40.447142 30562 consensus_queue.cc:260] T 5996cbfea86e4ef59ce6ee492d4c989f P abb560de64b04b9298c53a850739722d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cb2976444114a38bdd7891bea5f0c31" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 42129 } } peers { permanent_uuid: "951c99ff621647d5bd03cc2439ad069f" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 44505 } } peers { permanent_uuid: "abb560de64b04b9298c53a850739722d" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 39797 } }
I20260923 23:14:40.447314 30560 ts_tablet_manager.cc:1428] T 5996cbfea86e4ef59ce6ee492d4c989f P 6cb2976444114a38bdd7891bea5f0c31: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260923 23:14:40.447326 30545 heartbeater.cc:503] Master 127.24.227.254:41089 was elected leader, sending a full tablet report...
I20260923 23:14:40.447382 30562 ts_tablet_manager.cc:1428] T 5996cbfea86e4ef59ce6ee492d4c989f P abb560de64b04b9298c53a850739722d: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260923 23:14:40.447492 30414 heartbeater.cc:503] Master 127.24.227.254:41089 was elected leader, sending a full tablet report...
W20260923 23:14:40.464303 30284 tablet.cc:2369] T 5996cbfea86e4ef59ce6ee492d4c989f P 951c99ff621647d5bd03cc2439ad069f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 23:14:40.536742 30567 raft_consensus.cc:482] T 5996cbfea86e4ef59ce6ee492d4c989f P abb560de64b04b9298c53a850739722d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 23:14:40.536846 30567 raft_consensus.cc:504] T 5996cbfea86e4ef59ce6ee492d4c989f P abb560de64b04b9298c53a850739722d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cb2976444114a38bdd7891bea5f0c31" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 42129 } } peers { permanent_uuid: "951c99ff621647d5bd03cc2439ad069f" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 44505 } } peers { permanent_uuid: "abb560de64b04b9298c53a850739722d" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 39797 } }
I20260923 23:14:40.537184 30567 leader_election.cc:288] T 5996cbfea86e4ef59ce6ee492d4c989f P abb560de64b04b9298c53a850739722d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6cb2976444114a38bdd7891bea5f0c31 (127.24.227.195:42129), 951c99ff621647d5bd03cc2439ad069f (127.24.227.193:44505)
I20260923 23:14:40.538514 30416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:40.537338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.195:42129 (local address 127.24.227.194:44457)
0923 23:14:40.537402 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:40.537414 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:40.537455 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:40.537614 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:40.537615 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:40.537618 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:14:40.538091 (+ 473us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 23:14:40.538092 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:14:40.538100 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 23:14:40.538112 (+ 12us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:14:40.538357 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:14:40.538358 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:14:40.538359 (+ 1us) client_negotiation.cc:769] Sending connection context
0923 23:14:40.538410 (+ 51us) client_negotiation.cc:240] Negotiation successful
0923 23:14:40.538423 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260923 23:14:40.538573 30556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:40.537441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.194:44457 (local address 127.24.227.195:42129)
0923 23:14:40.537498 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:40.537500 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:40.537507 (+ 7us) server_negotiation.cc:407] Connection header received
0923 23:14:40.537532 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:40.537533 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:40.537539 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:40.537608 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:40.538221 (+ 613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:40.538222 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:40.538222 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 23:14:40.538240 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:40.538259 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 23:14:40.538264 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:40.538332 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:40.538488 (+ 156us) server_negotiation.cc:299] Negotiation successful
0923 23:14:40.538504 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260923 23:14:40.538623 30558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:40.537452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.194:35115 (local address 127.24.227.193:44505)
0923 23:14:40.537497 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:40.537499 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:40.537656 (+ 157us) server_negotiation.cc:407] Connection header received
0923 23:14:40.537716 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:40.537717 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:40.537723 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:40.537798 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:40.538376 (+ 578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:40.538377 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:40.538378 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 23:14:40.538383 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:40.538401 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 23:14:40.538405 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:40.538456 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:40.538537 (+ 81us) server_negotiation.cc:299] Negotiation successful
0923 23:14:40.538551 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260923 23:14:40.538632 30568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:40.537340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.193:44505 (local address 127.24.227.194:35115)
0923 23:14:40.537582 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:40.537591 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:40.537659 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:40.537801 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:40.537802 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:40.537820 (+ 18us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:14:40.538275 (+ 455us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 23:14:40.538276 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:14:40.538282 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 23:14:40.538293 (+ 11us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:14:40.538483 (+ 190us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:14:40.538484 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:14:40.538484 (+ 0us) client_negotiation.cc:769] Sending connection context
0923 23:14:40.538536 (+ 52us) client_negotiation.cc:240] Negotiation successful
0923 23:14:40.538547 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218,"mutex_wait_us":51,"thread_start_us":135,"threads_started":1}
I20260923 23:14:40.538975 30237 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5996cbfea86e4ef59ce6ee492d4c989f" candidate_uuid: "abb560de64b04b9298c53a850739722d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "951c99ff621647d5bd03cc2439ad069f" is_pre_election: true
I20260923 23:14:40.539022 30499 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5996cbfea86e4ef59ce6ee492d4c989f" candidate_uuid: "abb560de64b04b9298c53a850739722d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cb2976444114a38bdd7891bea5f0c31" is_pre_election: true
I20260923 23:14:40.539093 30237 raft_consensus.cc:2454] T 5996cbfea86e4ef59ce6ee492d4c989f P 951c99ff621647d5bd03cc2439ad069f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate abb560de64b04b9298c53a850739722d in term 0.
I20260923 23:14:40.539175 30499 raft_consensus.cc:2454] T 5996cbfea86e4ef59ce6ee492d4c989f P 6cb2976444114a38bdd7891bea5f0c31 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate abb560de64b04b9298c53a850739722d in term 0.
I20260923 23:14:40.539341 30303 leader_election.cc:302] T 5996cbfea86e4ef59ce6ee492d4c989f P abb560de64b04b9298c53a850739722d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 951c99ff621647d5bd03cc2439ad069f, abb560de64b04b9298c53a850739722d; no voters:
I20260923 23:14:40.539498 30567 raft_consensus.cc:2781] T 5996cbfea86e4ef59ce6ee492d4c989f P abb560de64b04b9298c53a850739722d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 23:14:40.539549 30567 raft_consensus.cc:482] T 5996cbfea86e4ef59ce6ee492d4c989f P abb560de64b04b9298c53a850739722d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 23:14:40.539592 30567 raft_consensus.cc:3037] T 5996cbfea86e4ef59ce6ee492d4c989f P abb560de64b04b9298c53a850739722d [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:14:40.540333 30567 raft_consensus.cc:504] T 5996cbfea86e4ef59ce6ee492d4c989f P abb560de64b04b9298c53a850739722d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cb2976444114a38bdd7891bea5f0c31" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 42129 } } peers { permanent_uuid: "951c99ff621647d5bd03cc2439ad069f" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 44505 } } peers { permanent_uuid: "abb560de64b04b9298c53a850739722d" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 39797 } }
I20260923 23:14:40.540489 30567 leader_election.cc:288] T 5996cbfea86e4ef59ce6ee492d4c989f P abb560de64b04b9298c53a850739722d [CANDIDATE]: Term 1 election: Requested vote from peers 6cb2976444114a38bdd7891bea5f0c31 (127.24.227.195:42129), 951c99ff621647d5bd03cc2439ad069f (127.24.227.193:44505)
I20260923 23:14:40.540592 30499 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5996cbfea86e4ef59ce6ee492d4c989f" candidate_uuid: "abb560de64b04b9298c53a850739722d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cb2976444114a38bdd7891bea5f0c31"
I20260923 23:14:40.540614 30237 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5996cbfea86e4ef59ce6ee492d4c989f" candidate_uuid: "abb560de64b04b9298c53a850739722d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "951c99ff621647d5bd03cc2439ad069f"
I20260923 23:14:40.540663 30499 raft_consensus.cc:3037] T 5996cbfea86e4ef59ce6ee492d4c989f P 6cb2976444114a38bdd7891bea5f0c31 [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:14:40.540675 30237 raft_consensus.cc:3037] T 5996cbfea86e4ef59ce6ee492d4c989f P 951c99ff621647d5bd03cc2439ad069f [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:14:40.541527 30237 raft_consensus.cc:2454] T 5996cbfea86e4ef59ce6ee492d4c989f P 951c99ff621647d5bd03cc2439ad069f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate abb560de64b04b9298c53a850739722d in term 1.
I20260923 23:14:40.541527 30499 raft_consensus.cc:2454] T 5996cbfea86e4ef59ce6ee492d4c989f P 6cb2976444114a38bdd7891bea5f0c31 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate abb560de64b04b9298c53a850739722d in term 1.
I20260923 23:14:40.541682 30303 leader_election.cc:302] T 5996cbfea86e4ef59ce6ee492d4c989f P abb560de64b04b9298c53a850739722d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 951c99ff621647d5bd03cc2439ad069f, abb560de64b04b9298c53a850739722d; no voters:
I20260923 23:14:40.541795 30567 raft_consensus.cc:2781] T 5996cbfea86e4ef59ce6ee492d4c989f P abb560de64b04b9298c53a850739722d [term 1 FOLLOWER]: Leader election won for term 1
I20260923 23:14:40.541929 30567 raft_consensus.cc:686] T 5996cbfea86e4ef59ce6ee492d4c989f P abb560de64b04b9298c53a850739722d [term 1 LEADER]: Becoming Leader. State: Replica: abb560de64b04b9298c53a850739722d, State: Running, Role: LEADER
I20260923 23:14:40.542037 30567 consensus_queue.cc:237] T 5996cbfea86e4ef59ce6ee492d4c989f P abb560de64b04b9298c53a850739722d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cb2976444114a38bdd7891bea5f0c31" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 42129 } } peers { permanent_uuid: "951c99ff621647d5bd03cc2439ad069f" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 44505 } } peers { permanent_uuid: "abb560de64b04b9298c53a850739722d" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 39797 } }
I20260923 23:14:40.542908 30095 catalog_manager.cc:5495] T 5996cbfea86e4ef59ce6ee492d4c989f P abb560de64b04b9298c53a850739722d reported cstate change: term changed from 0 to 1, leader changed from <none> to abb560de64b04b9298c53a850739722d (127.24.227.194). New cstate: current_term: 1 leader_uuid: "abb560de64b04b9298c53a850739722d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cb2976444114a38bdd7891bea5f0c31" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 42129 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "951c99ff621647d5bd03cc2439ad069f" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 44505 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "abb560de64b04b9298c53a850739722d" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 39797 } health_report { overall_health: HEALTHY } } }
I20260923 23:14:40.550740 30557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:40.550014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59530 (local address 127.24.227.194:39797)
0923 23:14:40.550062 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:40.550064 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:40.550072 (+ 8us) server_negotiation.cc:407] Connection header received
0923 23:14:40.550115 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:40.550116 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:40.550124 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:40.550160 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:40.550590 (+ 430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:40.550591 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:40.550591 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 23:14:40.550594 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:40.550605 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 23:14:40.550608 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:40.550651 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:40.550692 (+ 41us) server_negotiation.cc:299] Negotiation successful
0923 23:14:40.550701 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260923 23:14:40.552276 30237 raft_consensus.cc:1261] T 5996cbfea86e4ef59ce6ee492d4c989f P 951c99ff621647d5bd03cc2439ad069f [term 1 FOLLOWER]: Refusing update from remote peer abb560de64b04b9298c53a850739722d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 23:14:40.552276 30499 raft_consensus.cc:1261] T 5996cbfea86e4ef59ce6ee492d4c989f P 6cb2976444114a38bdd7891bea5f0c31 [term 1 FOLLOWER]: Refusing update from remote peer abb560de64b04b9298c53a850739722d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 23:14:40.552683 30569 consensus_queue.cc:1035] T 5996cbfea86e4ef59ce6ee492d4c989f P abb560de64b04b9298c53a850739722d [LEADER]: Connected to new peer: Peer: permanent_uuid: "951c99ff621647d5bd03cc2439ad069f" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 44505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 23:14:40.552805 30567 consensus_queue.cc:1035] T 5996cbfea86e4ef59ce6ee492d4c989f P abb560de64b04b9298c53a850739722d [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cb2976444114a38bdd7891bea5f0c31" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 42129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 23:14:40.555012 30573 mvcc.cc:205] Tried to move back new op lower bound from 7332680829139435520 to 7332680829100785664. Current Snapshot: MvccSnapshot[applied={T|T < 7332680829139435520}]
I20260923 23:14:40.556783 30575 mvcc.cc:205] Tried to move back new op lower bound from 7332680829139435520 to 7332680829100785664. Current Snapshot: MvccSnapshot[applied={T|T < 7332680829139435520 or (T in {7332680829139435520})}]
I20260923 23:14:40.557675 30094 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48782:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 23:14:40.557756 30094 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48782:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 23:14:40.558434 30094 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6234ebf4e249434f8aaef32b1c905890: Sending DeleteTablet for 3 replicas of tablet 5996cbfea86e4ef59ce6ee492d4c989f
I20260923 23:14:40.558776 30348 tablet_service.cc:1514] Processing DeleteTablet for tablet 5996cbfea86e4ef59ce6ee492d4c989f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 23:14:40 UTC) from {username='slave'} at 127.0.0.1:59524
I20260923 23:14:40.558790 30479 tablet_service.cc:1514] Processing DeleteTablet for tablet 5996cbfea86e4ef59ce6ee492d4c989f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 23:14:40 UTC) from {username='slave'} at 127.0.0.1:40964
I20260923 23:14:40.559010 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 30155
I20260923 23:14:40.558990 30217 tablet_service.cc:1514] Processing DeleteTablet for tablet 5996cbfea86e4ef59ce6ee492d4c989f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 23:14:40 UTC) from {username='slave'} at 127.0.0.1:46256
I20260923 23:14:40.561069 30581 tablet_replica.cc:330] T 5996cbfea86e4ef59ce6ee492d4c989f P abb560de64b04b9298c53a850739722d: stopping tablet replica
I20260923 23:14:40.561197 30581 raft_consensus.cc:2229] T 5996cbfea86e4ef59ce6ee492d4c989f P abb560de64b04b9298c53a850739722d [term 1 LEADER]: Raft consensus shutting down.
I20260923 23:14:40.561321 30581 raft_consensus.cc:2258] T 5996cbfea86e4ef59ce6ee492d4c989f P abb560de64b04b9298c53a850739722d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 23:14:40.561571 30582 tablet_replica.cc:330] T 5996cbfea86e4ef59ce6ee492d4c989f P 6cb2976444114a38bdd7891bea5f0c31: stopping tablet replica
I20260923 23:14:40.561704 30582 raft_consensus.cc:2229] T 5996cbfea86e4ef59ce6ee492d4c989f P 6cb2976444114a38bdd7891bea5f0c31 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 23:14:40.561841 30582 raft_consensus.cc:2258] T 5996cbfea86e4ef59ce6ee492d4c989f P 6cb2976444114a38bdd7891bea5f0c31 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 23:14:40.561901 30581 ts_tablet_manager.cc:1905] T 5996cbfea86e4ef59ce6ee492d4c989f P abb560de64b04b9298c53a850739722d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 23:14:40.562359 30582 ts_tablet_manager.cc:1905] T 5996cbfea86e4ef59ce6ee492d4c989f P 6cb2976444114a38bdd7891bea5f0c31: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 23:14:40.563725 30581 ts_tablet_manager.cc:1918] T 5996cbfea86e4ef59ce6ee492d4c989f P abb560de64b04b9298c53a850739722d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 23:14:40.563800 30581 log.cc:1196] T 5996cbfea86e4ef59ce6ee492d4c989f P abb560de64b04b9298c53a850739722d: Deleting WAL directory at /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790205259943518-25487-0/minicluster-data/ts-1/wal/wals/5996cbfea86e4ef59ce6ee492d4c989f
I20260923 23:14:40.564119 30581 ts_tablet_manager.cc:1939] T 5996cbfea86e4ef59ce6ee492d4c989f P abb560de64b04b9298c53a850739722d: Deleting consensus metadata
I20260923 23:14:40.564482 30582 ts_tablet_manager.cc:1918] T 5996cbfea86e4ef59ce6ee492d4c989f P 6cb2976444114a38bdd7891bea5f0c31: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 23:14:40.564601 30582 log.cc:1196] T 5996cbfea86e4ef59ce6ee492d4c989f P 6cb2976444114a38bdd7891bea5f0c31: Deleting WAL directory at /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790205259943518-25487-0/minicluster-data/ts-2/wal/wals/5996cbfea86e4ef59ce6ee492d4c989f
I20260923 23:14:40.564585 30082 catalog_manager.cc:4841] TS abb560de64b04b9298c53a850739722d (127.24.227.194:39797): tablet 5996cbfea86e4ef59ce6ee492d4c989f (table test-table [id=4d88b0492adb4c37a8f6f3b8079d8749]) successfully deleted
W20260923 23:14:40.564905 30081 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.227.193:44505 (error 108)
I20260923 23:14:40.564917 30582 ts_tablet_manager.cc:1939] T 5996cbfea86e4ef59ce6ee492d4c989f P 6cb2976444114a38bdd7891bea5f0c31: Deleting consensus metadata
I20260923 23:14:40.565148 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 30286
I20260923 23:14:40.565798 30555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:40.565227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.193:44505 (local address 127.0.0.1:46272)
0923 23:14:40.565716 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:40.565761 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.193:44505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":468}
I20260923 23:14:40.566102 30079 catalog_manager.cc:4841] TS 6cb2976444114a38bdd7891bea5f0c31 (127.24.227.195:42129): tablet 5996cbfea86e4ef59ce6ee492d4c989f (table test-table [id=4d88b0492adb4c37a8f6f3b8079d8749]) successfully deleted
W20260923 23:14:40.566372 30081 catalog_manager.cc:4584] TS 951c99ff621647d5bd03cc2439ad069f (127.24.227.193:44505): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5996cbfea86e4ef59ce6ee492d4c989f: Network error: Client connection negotiation failed: client connection to 127.24.227.193:44505: connect: Connection refused (error 111)
I20260923 23:14:40.569149 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 30417
I20260923 23:14:40.573413 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 30064
2026-09-23T23:14:40Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (629 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260923 23:14:40.592036 25487 mini_oidc.cc:152] Starting JWKS server
I20260923 23:14:40.593035 25487 webserver.cc:460] Webserver started at https://127.0.0.1:39811/ using document root <none> and password file <none>
I20260923 23:14:40.593137 25487 mini_oidc.cc:189] Starting OIDC Discovery server
I20260923 23:14:40.593327 25487 webserver.cc:460] Webserver started at http://127.0.0.1:42095/ using document root <none> and password file <none>
2026-09-23T23:14:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T23:14:40Z Disabled control of system clock
I20260923 23:14:40.600253 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790205259943518-25487-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790205259943518-25487-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790205259943518-25487-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.254:41717
--webserver_interface=127.24.227.254
--webserver_port=0
--builtin_ntp_servers=127.24.227.212:42097
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.227.254:41717
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39811/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790205259943518-25487-0/testchainca.cert with env {}
W20260923 23:14:40.675443 30589 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:14:40.675626 30589 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:14:40.675648 30589 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:14:40.676739 30589 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 23:14:40.676775 30589 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:14:40.676788 30589 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:14:40.676800 30589 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 23:14:40.676812 30589 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260923 23:14:40.676824 30589 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260923 23:14:40.676836 30589 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:39811/jwks.json
I20260923 23:14:40.678244 30589 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:42097
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790205259943518-25487-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790205259943518-25487-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.227.254:41717
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.254:41717
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39811/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.227.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790205259943518-25487-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30589
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790205259943518-25487-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:14:40.678412 30589 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:14:40.678550 30589 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:14:40.682828 30597 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:40.682827 30595 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:14:40.683029 30589 server_base.cc:1034] running on GCE node
W20260923 23:14:40.682894 30594 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:14:40.684228 30589 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:14:40.684494 30589 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:14:40.685659 30589 hybrid_clock.cc:648] HybridClock initialized: now 1790205280685647 us; error 44 us; skew 500 ppm
I20260923 23:14:40.686791 30589 webserver.cc:460] Webserver started at http://127.24.227.254:33425/ using document root <none> and password file <none>
I20260923 23:14:40.687004 30589 fs_manager.cc:359] Metadata directory not provided
I20260923 23:14:40.687071 30589 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:14:40.687213 30589 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:14:40.688237 30589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790205259943518-25487-0/minicluster-data/master-0/data/instance:
uuid: "774d4c2082b54985b9cc0d2db206763a"
format_stamp: "Formatted at 2026-09-23 23:14:40 on dist-test-slave-754n"
server_key: "1ed351505cb92c6bba4abab5f20551c2"
server_key_iv: "76fcfe5d6be6c0829622b778c6647513"
server_key_version: "encryptionkey@0"
I20260923 23:14:40.688606 30589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790205259943518-25487-0/minicluster-data/master-0/wal/instance:
uuid: "774d4c2082b54985b9cc0d2db206763a"
format_stamp: "Formatted at 2026-09-23 23:14:40 on dist-test-slave-754n"
server_key: "1ed351505cb92c6bba4abab5f20551c2"
server_key_iv: "76fcfe5d6be6c0829622b778c6647513"
server_key_version: "encryptionkey@0"
I20260923 23:14:40.689836 30589 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 23:14:40.690478 30603 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:40.690639 30589 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 23:14:40.690766 30589 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790205259943518-25487-0/minicluster-data/master-0/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790205259943518-25487-0/minicluster-data/master-0/wal
uuid: "774d4c2082b54985b9cc0d2db206763a"
format_stamp: "Formatted at 2026-09-23 23:14:40 on dist-test-slave-754n"
server_key: "1ed351505cb92c6bba4abab5f20551c2"
server_key_iv: "76fcfe5d6be6c0829622b778c6647513"
server_key_version: "encryptionkey@0"
I20260923 23:14:40.690840 30589 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790205259943518-25487-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790205259943518-25487-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790205259943518-25487-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:14:40.710157 30589 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:14:40.710424 30589 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:14:40.710575 30589 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:14:40.714278 30589 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.254:41717
I20260923 23:14:40.714313 30655 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.254:41717 every 8 connection(s)
I20260923 23:14:40.714634 30589 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
I20260923 23:14:40.715440 30656 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:14:40.717558 30656 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 774d4c2082b54985b9cc0d2db206763a: Bootstrap starting.
I20260923 23:14:40.718122 30656 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 774d4c2082b54985b9cc0d2db206763a: Neither blocks nor log segments found. Creating new log.
I20260923 23:14:40.718394 30656 log.cc:824] T 00000000000000000000000000000000 P 774d4c2082b54985b9cc0d2db206763a: Log is configured to *not* fsync() on all Append() calls
I20260923 23:14:40.718973 30656 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 774d4c2082b54985b9cc0d2db206763a: No bootstrap required, opened a new log
I20260923 23:14:40.720453 30656 raft_consensus.cc:348] T 00000000000000000000000000000000 P 774d4c2082b54985b9cc0d2db206763a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "774d4c2082b54985b9cc0d2db206763a" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41717 } }
I20260923 23:14:40.720610 30656 raft_consensus.cc:374] T 00000000000000000000000000000000 P 774d4c2082b54985b9cc0d2db206763a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:14:40.720633 30656 raft_consensus.cc:729] T 00000000000000000000000000000000 P 774d4c2082b54985b9cc0d2db206763a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 774d4c2082b54985b9cc0d2db206763a, State: Initialized, Role: FOLLOWER
I20260923 23:14:40.720819 30656 consensus_queue.cc:260] T 00000000000000000000000000000000 P 774d4c2082b54985b9cc0d2db206763a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "774d4c2082b54985b9cc0d2db206763a" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41717 } }
I20260923 23:14:40.720893 30656 raft_consensus.cc:388] T 00000000000000000000000000000000 P 774d4c2082b54985b9cc0d2db206763a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 23:14:40.720933 30656 raft_consensus.cc:482] T 00000000000000000000000000000000 P 774d4c2082b54985b9cc0d2db206763a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 23:14:40.720979 30656 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 774d4c2082b54985b9cc0d2db206763a [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:14:40.721438 30656 raft_consensus.cc:504] T 00000000000000000000000000000000 P 774d4c2082b54985b9cc0d2db206763a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "774d4c2082b54985b9cc0d2db206763a" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41717 } }
I20260923 23:14:40.721585 30656 leader_election.cc:302] T 00000000000000000000000000000000 P 774d4c2082b54985b9cc0d2db206763a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 774d4c2082b54985b9cc0d2db206763a; no voters:
I20260923 23:14:40.721760 30656 leader_election.cc:288] T 00000000000000000000000000000000 P 774d4c2082b54985b9cc0d2db206763a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 23:14:40.721807 30659 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 774d4c2082b54985b9cc0d2db206763a [term 1 FOLLOWER]: Leader election won for term 1
I20260923 23:14:40.721972 30659 raft_consensus.cc:686] T 00000000000000000000000000000000 P 774d4c2082b54985b9cc0d2db206763a [term 1 LEADER]: Becoming Leader. State: Replica: 774d4c2082b54985b9cc0d2db206763a, State: Running, Role: LEADER
I20260923 23:14:40.722003 30656 sys_catalog.cc:564] T 00000000000000000000000000000000 P 774d4c2082b54985b9cc0d2db206763a [sys.catalog]: configured and running, proceeding with master startup.
I20260923 23:14:40.722100 30659 consensus_queue.cc:237] T 00000000000000000000000000000000 P 774d4c2082b54985b9cc0d2db206763a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "774d4c2082b54985b9cc0d2db206763a" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41717 } }
I20260923 23:14:40.722555 30660 sys_catalog.cc:455] T 00000000000000000000000000000000 P 774d4c2082b54985b9cc0d2db206763a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "774d4c2082b54985b9cc0d2db206763a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "774d4c2082b54985b9cc0d2db206763a" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41717 } } }
I20260923 23:14:40.722759 30661 sys_catalog.cc:455] T 00000000000000000000000000000000 P 774d4c2082b54985b9cc0d2db206763a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 774d4c2082b54985b9cc0d2db206763a. Latest consensus state: current_term: 1 leader_uuid: "774d4c2082b54985b9cc0d2db206763a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "774d4c2082b54985b9cc0d2db206763a" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41717 } } }
I20260923 23:14:40.722919 30661 sys_catalog.cc:458] T 00000000000000000000000000000000 P 774d4c2082b54985b9cc0d2db206763a [sys.catalog]: This master's current role is: LEADER
I20260923 23:14:40.722868 30660 sys_catalog.cc:458] T 00000000000000000000000000000000 P 774d4c2082b54985b9cc0d2db206763a [sys.catalog]: This master's current role is: LEADER
I20260923 23:14:40.723127 30669 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 23:14:40.723611 30669 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 23:14:40.723706 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 30589
I20260923 23:14:40.723800 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790205259943518-25487-0/minicluster-data/master-0/wal/instance
I20260923 23:14:40.723999 25487 external_mini_cluster.cc:1428] Setting key 34f97b7a769306419060909fd82f7be8
I20260923 23:14:40.725289 30669 catalog_manager.cc:1347] Generated new cluster ID: a57cda669656462eb6480423128820f6
I20260923 23:14:40.725347 30669 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 23:14:40.730065 30678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:40.724389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50586 (local address 127.24.227.254:41717)
0923 23:14:40.725005 (+ 616us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:40.725013 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:40.725044 (+ 31us) server_negotiation.cc:407] Connection header received
0923 23:14:40.725149 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:40.725156 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:40.725228 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:40.725363 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:40.725666 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:40.726979 (+ 1313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:40.728034 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:40.728681 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:40.728801 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:40.729386 (+ 585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:40.729389 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:40.729390 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 23:14:40.729593 (+ 203us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:40.729645 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 23:14:40.729659 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:40.729725 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:40.729796 (+ 71us) server_negotiation.cc:299] Negotiation successful
0923 23:14:40.729865 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":517,"thread_start_us":89,"threads_started":1}
I20260923 23:14:40.730978 25487 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 23:14:40.739601 30669 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 23:14:40.740087 30669 catalog_manager.cc:1504] Loading token signing keys...
I20260923 23:14:40.753978 30669 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 774d4c2082b54985b9cc0d2db206763a: Generated new TSK 0
I20260923 23:14:40.754135 30669 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260923 23:14:40.782186 30686 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260923 23:14:40.782269 30686 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260923 23:14:40.782466 30678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:40.764759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50598 (local address 127.24.227.254:41717)
0923 23:14:40.764816 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:40.764818 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:40.764827 (+ 9us) server_negotiation.cc:407] Connection header received
0923 23:14:40.764895 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:40.764895 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:40.764909 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:40.764990 (+ 81us) server_negotiation.cc:226] Negotiated authn=JWT
0923 23:14:40.765139 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:40.765916 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:40.767086 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:40.767397 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:40.767479 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:40.767718 (+ 239us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0923 23:14:40.782285 (+ 14567us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39811/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0923 23:14:40.782407 (+ 122us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:50598: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39811/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":39}
W20260923 23:14:40.782521 30678 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:50598: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39811/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260923 23:14:40.783298 30685 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 23:14:40.764547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41717 (local address 127.0.0.1:50598)
0923 23:14:40.764733 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:40.764749 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:40.764811 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:40.764996 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:40.764996 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:40.764998 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0923 23:14:40.765059 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:40.765062 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:40.766051 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:40.766052 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:40.766999 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:40.767002 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:40.767489 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:40.767491 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:40.767581 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:40.767601 (+ 20us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0923 23:14:40.782895 (+ 15294us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39811/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0923 23:14:40.783001 (+ 106us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.24.227.254:41717: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39811/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":95,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.24.227.254:41717: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39811/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260923 23:14:40.783849 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 30589
2026-09-23T23:14:40Z chronyd exiting
I20260923 23:14:40.820194 25487 test_util.cc:175] -----------------------------------------------
I20260923 23:14:40.820272 25487 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790205259943518-25487-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (229 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260923 23:14:40.820945 25487 mini_oidc.cc:152] Starting JWKS server
I20260923 23:14:40.821616 25487 webserver.cc:460] Webserver started at https://127.0.0.1:44335/ using document root <none> and password file <none>
I20260923 23:14:40.821682 25487 mini_oidc.cc:189] Starting OIDC Discovery server
I20260923 23:14:40.821857 25487 webserver.cc:460] Webserver started at http://127.0.0.1:44541/ using document root <none> and password file <none>
2026-09-23T23:14:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T23:14:40Z Disabled control of system clock
I20260923 23:14:40.828831 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790205259943518-25487-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790205259943518-25487-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790205259943518-25487-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.254:35607
--webserver_interface=127.24.227.254
--webserver_port=0
--builtin_ntp_servers=127.24.227.212:38225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.227.254:35607
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44335/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790205259943518-25487-0/testchainca.cert with env {}
W20260923 23:14:40.904165 30692 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:14:40.904358 30692 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:14:40.904378 30692 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:14:40.905400 30692 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 23:14:40.905431 30692 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:14:40.905444 30692 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:14:40.905457 30692 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 23:14:40.905467 30692 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260923 23:14:40.905479 30692 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260923 23:14:40.905490 30692 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:44335/jwks.json
I20260923 23:14:40.906831 30692 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:38225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790205259943518-25487-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790205259943518-25487-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.227.254:35607
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.254:35607
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44335/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.227.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790205259943518-25487-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30692
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790205259943518-25487-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:14:40.907007 30692 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:14:40.907159 30692 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:14:40.911448 30698 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:14:40.911504 30692 server_base.cc:1034] running on GCE node
W20260923 23:14:40.911509 30700 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:40.911456 30697 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:14:40.911962 30692 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:14:40.912226 30692 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:14:40.913362 30692 hybrid_clock.cc:648] HybridClock initialized: now 1790205280913349 us; error 28 us; skew 500 ppm
I20260923 23:14:40.917496 30692 webserver.cc:460] Webserver started at http://127.24.227.254:36467/ using document root <none> and password file <none>
I20260923 23:14:40.917733 30692 fs_manager.cc:359] Metadata directory not provided
I20260923 23:14:40.917778 30692 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:14:40.917933 30692 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:14:40.918910 30692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790205259943518-25487-0/minicluster-data/master-0/data/instance:
uuid: "d2bd3c72d04b4b3aa3e81c8ab26e5632"
format_stamp: "Formatted at 2026-09-23 23:14:40 on dist-test-slave-754n"
server_key: "93819056b8b57f0cd3ad38e43dada48d"
server_key_iv: "49d7735ea8f4fa5ed5329d88d166575f"
server_key_version: "encryptionkey@0"
I20260923 23:14:40.919238 30692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790205259943518-25487-0/minicluster-data/master-0/wal/instance:
uuid: "d2bd3c72d04b4b3aa3e81c8ab26e5632"
format_stamp: "Formatted at 2026-09-23 23:14:40 on dist-test-slave-754n"
server_key: "93819056b8b57f0cd3ad38e43dada48d"
server_key_iv: "49d7735ea8f4fa5ed5329d88d166575f"
server_key_version: "encryptionkey@0"
I20260923 23:14:40.920490 30692 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 23:14:40.921155 30706 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:40.921327 30692 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 23:14:40.921406 30692 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790205259943518-25487-0/minicluster-data/master-0/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790205259943518-25487-0/minicluster-data/master-0/wal
uuid: "d2bd3c72d04b4b3aa3e81c8ab26e5632"
format_stamp: "Formatted at 2026-09-23 23:14:40 on dist-test-slave-754n"
server_key: "93819056b8b57f0cd3ad38e43dada48d"
server_key_iv: "49d7735ea8f4fa5ed5329d88d166575f"
server_key_version: "encryptionkey@0"
I20260923 23:14:40.921456 30692 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790205259943518-25487-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790205259943518-25487-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790205259943518-25487-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:14:40.931020 30692 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:14:40.931222 30692 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:14:40.931324 30692 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:14:40.934919 30692 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.254:35607
I20260923 23:14:40.934960 30758 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.254:35607 every 8 connection(s)
I20260923 23:14:40.935223 30692 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
I20260923 23:14:40.935796 30759 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:14:40.938227 30759 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2bd3c72d04b4b3aa3e81c8ab26e5632: Bootstrap starting.
I20260923 23:14:40.938745 30759 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d2bd3c72d04b4b3aa3e81c8ab26e5632: Neither blocks nor log segments found. Creating new log.
I20260923 23:14:40.939013 30759 log.cc:824] T 00000000000000000000000000000000 P d2bd3c72d04b4b3aa3e81c8ab26e5632: Log is configured to *not* fsync() on all Append() calls
I20260923 23:14:40.939625 30759 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2bd3c72d04b4b3aa3e81c8ab26e5632: No bootstrap required, opened a new log
I20260923 23:14:40.941118 30759 raft_consensus.cc:348] T 00000000000000000000000000000000 P d2bd3c72d04b4b3aa3e81c8ab26e5632 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2bd3c72d04b4b3aa3e81c8ab26e5632" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 35607 } }
I20260923 23:14:40.941254 30759 raft_consensus.cc:374] T 00000000000000000000000000000000 P d2bd3c72d04b4b3aa3e81c8ab26e5632 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:14:40.941290 30759 raft_consensus.cc:729] T 00000000000000000000000000000000 P d2bd3c72d04b4b3aa3e81c8ab26e5632 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2bd3c72d04b4b3aa3e81c8ab26e5632, State: Initialized, Role: FOLLOWER
I20260923 23:14:40.941434 30759 consensus_queue.cc:260] T 00000000000000000000000000000000 P d2bd3c72d04b4b3aa3e81c8ab26e5632 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2bd3c72d04b4b3aa3e81c8ab26e5632" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 35607 } }
I20260923 23:14:40.941538 30759 raft_consensus.cc:388] T 00000000000000000000000000000000 P d2bd3c72d04b4b3aa3e81c8ab26e5632 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 23:14:40.941581 30759 raft_consensus.cc:482] T 00000000000000000000000000000000 P d2bd3c72d04b4b3aa3e81c8ab26e5632 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 23:14:40.941632 30759 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d2bd3c72d04b4b3aa3e81c8ab26e5632 [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:14:40.942194 30759 raft_consensus.cc:504] T 00000000000000000000000000000000 P d2bd3c72d04b4b3aa3e81c8ab26e5632 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2bd3c72d04b4b3aa3e81c8ab26e5632" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 35607 } }
I20260923 23:14:40.942310 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 30692
I20260923 23:14:40.942312 30759 leader_election.cc:302] T 00000000000000000000000000000000 P d2bd3c72d04b4b3aa3e81c8ab26e5632 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d2bd3c72d04b4b3aa3e81c8ab26e5632; no voters:
I20260923 23:14:40.942409 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790205259943518-25487-0/minicluster-data/master-0/wal/instance
I20260923 23:14:40.942476 30759 leader_election.cc:288] T 00000000000000000000000000000000 P d2bd3c72d04b4b3aa3e81c8ab26e5632 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 23:14:40.942545 30762 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d2bd3c72d04b4b3aa3e81c8ab26e5632 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 23:14:40.942617 25487 external_mini_cluster.cc:1428] Setting key b9abba7c929f5526f98712ce17878ea7
I20260923 23:14:40.942687 30762 raft_consensus.cc:686] T 00000000000000000000000000000000 P d2bd3c72d04b4b3aa3e81c8ab26e5632 [term 1 LEADER]: Becoming Leader. State: Replica: d2bd3c72d04b4b3aa3e81c8ab26e5632, State: Running, Role: LEADER
I20260923 23:14:40.942713 30759 sys_catalog.cc:564] T 00000000000000000000000000000000 P d2bd3c72d04b4b3aa3e81c8ab26e5632 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 23:14:40.942794 30762 consensus_queue.cc:237] T 00000000000000000000000000000000 P d2bd3c72d04b4b3aa3e81c8ab26e5632 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2bd3c72d04b4b3aa3e81c8ab26e5632" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 35607 } }
I20260923 23:14:40.943136 30762 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2bd3c72d04b4b3aa3e81c8ab26e5632 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d2bd3c72d04b4b3aa3e81c8ab26e5632" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2bd3c72d04b4b3aa3e81c8ab26e5632" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 35607 } } }
I20260923 23:14:40.943210 30762 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2bd3c72d04b4b3aa3e81c8ab26e5632 [sys.catalog]: This master's current role is: LEADER
I20260923 23:14:40.943179 30764 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2bd3c72d04b4b3aa3e81c8ab26e5632 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d2bd3c72d04b4b3aa3e81c8ab26e5632. Latest consensus state: current_term: 1 leader_uuid: "d2bd3c72d04b4b3aa3e81c8ab26e5632" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2bd3c72d04b4b3aa3e81c8ab26e5632" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 35607 } } }
I20260923 23:14:40.943265 30764 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2bd3c72d04b4b3aa3e81c8ab26e5632 [sys.catalog]: This master's current role is: LEADER
I20260923 23:14:40.944348 30769 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 23:14:40.944872 30769 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 23:14:40.947896 30769 catalog_manager.cc:1347] Generated new cluster ID: 5ed6e6ff52ea4338903c68d75f386e0e
I20260923 23:14:40.947944 30769 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 23:14:40.948350 30767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:40.943073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42042 (local address 127.24.227.254:35607)
0923 23:14:40.943439 (+ 366us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:40.943450 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:40.943588 (+ 138us) server_negotiation.cc:407] Connection header received
0923 23:14:40.943736 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:40.943744 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:40.943810 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:40.943961 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:40.944280 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:40.945507 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:40.946529 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:40.946921 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:40.947001 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:40.947633 (+ 632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:40.947636 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:40.947637 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 23:14:40.947817 (+ 180us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:40.947855 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 23:14:40.947893 (+ 38us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:40.947982 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:40.948125 (+ 143us) server_negotiation.cc:299] Negotiation successful
0923 23:14:40.948203 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":56,"threads_started":1}
I20260923 23:14:40.949144 25487 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 23:14:40.977933 30769 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 23:14:40.978389 30769 catalog_manager.cc:1504] Loading token signing keys...
I20260923 23:14:40.981726 30769 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d2bd3c72d04b4b3aa3e81c8ab26e5632: Generated new TSK 0
I20260923 23:14:40.981890 30769 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260923 23:14:40.999593 30789 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260923 23:14:40.999655 30789 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260923 23:14:40.999960 30767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:40.986537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42058 (local address 127.24.227.254:35607)
0923 23:14:40.986594 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:40.986595 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:40.986710 (+ 115us) server_negotiation.cc:407] Connection header received
0923 23:14:40.986770 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:40.986771 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:40.986781 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:40.986834 (+ 53us) server_negotiation.cc:226] Negotiated authn=JWT
0923 23:14:40.986973 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:40.987417 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:40.988479 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:40.988660 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:40.988712 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:40.988894 (+ 182us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0923 23:14:40.999762 (+ 10868us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44335/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0923 23:14:40.999859 (+ 97us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:42058: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44335/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":35}
W20260923 23:14:41.000013 30767 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:42058: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44335/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260923 23:14:41.000056 30788 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 23:14:40.986483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:35607 (local address 127.0.0.1:42058)
0923 23:14:40.986636 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:40.986648 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:40.986709 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:40.986853 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:40.986853 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:40.986856 (+ 3us) client_negotiation.cc:189] Negotiated authn=JWT
0923 23:14:40.986905 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:40.986907 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:40.987483 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:40.987484 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:40.988405 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:40.988407 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:40.988757 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:40.988758 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:40.988813 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:40.988818 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0923 23:14:40.999851 (+ 11033us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44335/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0923 23:14:40.999962 (+ 111us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.24.227.254:35607: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44335/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":105,"threads_started":1}
I20260923 23:14:41.000263 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 30692
2026-09-23T23:14:41Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (216 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260923 23:14:41.037731 25487 mini_oidc.cc:152] Starting JWKS server
I20260923 23:14:41.038383 25487 webserver.cc:460] Webserver started at https://127.0.0.1:38599/ using document root <none> and password file <none>
I20260923 23:14:41.038420 25487 mini_oidc.cc:189] Starting OIDC Discovery server
I20260923 23:14:41.038589 25487 webserver.cc:460] Webserver started at http://127.0.0.1:42335/ using document root <none> and password file <none>
2026-09-23T23:14:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T23:14:41Z Disabled control of system clock
I20260923 23:14:41.045866 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790205259943518-25487-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790205259943518-25487-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790205259943518-25487-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.254:44775
--webserver_interface=127.24.227.254
--webserver_port=0
--builtin_ntp_servers=127.24.227.212:35321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.227.254:44775
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38599/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260923 23:14:41.121908 30795 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:14:41.122092 30795 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:14:41.122113 30795 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:14:41.123275 30795 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 23:14:41.123314 30795 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:14:41.123327 30795 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:14:41.123340 30795 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 23:14:41.123353 30795 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260923 23:14:41.123402 30795 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260923 23:14:41.123418 30795 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:38599/jwks.json
I20260923 23:14:41.124751 30795 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:35321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790205259943518-25487-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790205259943518-25487-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.227.254:44775
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.254:44775
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38599/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.227.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30795
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790205259943518-25487-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:14:41.124985 30795 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:14:41.125274 30795 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:14:41.129686 30800 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:41.129693 30801 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:41.129699 30803 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:14:41.129915 30795 server_base.cc:1034] running on GCE node
I20260923 23:14:41.130151 30795 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:14:41.130415 30795 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:14:41.131573 30795 hybrid_clock.cc:648] HybridClock initialized: now 1790205281131555 us; error 27 us; skew 500 ppm
I20260923 23:14:41.132858 30795 webserver.cc:460] Webserver started at http://127.24.227.254:34337/ using document root <none> and password file <none>
I20260923 23:14:41.133116 30795 fs_manager.cc:359] Metadata directory not provided
I20260923 23:14:41.133184 30795 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:14:41.133328 30795 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:14:41.134302 30795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790205259943518-25487-0/minicluster-data/master-0/data/instance:
uuid: "b02a236d7f0149b392339d6a556aa0c3"
format_stamp: "Formatted at 2026-09-23 23:14:41 on dist-test-slave-754n"
server_key: "71a6eab682ab9ee4a04caf38cc98e94e"
server_key_iv: "fc9b1c499fef7bbcfbf5e053f63a9d6a"
server_key_version: "encryptionkey@0"
I20260923 23:14:41.134636 30795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790205259943518-25487-0/minicluster-data/master-0/wal/instance:
uuid: "b02a236d7f0149b392339d6a556aa0c3"
format_stamp: "Formatted at 2026-09-23 23:14:41 on dist-test-slave-754n"
server_key: "71a6eab682ab9ee4a04caf38cc98e94e"
server_key_iv: "fc9b1c499fef7bbcfbf5e053f63a9d6a"
server_key_version: "encryptionkey@0"
I20260923 23:14:41.135813 30795 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 23:14:41.136502 30809 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:41.136668 30795 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 23:14:41.136778 30795 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790205259943518-25487-0/minicluster-data/master-0/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790205259943518-25487-0/minicluster-data/master-0/wal
uuid: "b02a236d7f0149b392339d6a556aa0c3"
format_stamp: "Formatted at 2026-09-23 23:14:41 on dist-test-slave-754n"
server_key: "71a6eab682ab9ee4a04caf38cc98e94e"
server_key_iv: "fc9b1c499fef7bbcfbf5e053f63a9d6a"
server_key_version: "encryptionkey@0"
I20260923 23:14:41.136830 30795 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790205259943518-25487-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790205259943518-25487-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790205259943518-25487-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:14:41.145685 30795 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:14:41.145882 30795 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:14:41.145972 30795 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:14:41.149353 30795 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.254:44775
I20260923 23:14:41.149398 30861 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.254:44775 every 8 connection(s)
I20260923 23:14:41.149677 30795 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
I20260923 23:14:41.150269 30862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:14:41.152616 30862 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b02a236d7f0149b392339d6a556aa0c3: Bootstrap starting.
I20260923 23:14:41.153167 30862 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b02a236d7f0149b392339d6a556aa0c3: Neither blocks nor log segments found. Creating new log.
I20260923 23:14:41.153439 30862 log.cc:824] T 00000000000000000000000000000000 P b02a236d7f0149b392339d6a556aa0c3: Log is configured to *not* fsync() on all Append() calls
I20260923 23:14:41.154073 30862 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b02a236d7f0149b392339d6a556aa0c3: No bootstrap required, opened a new log
I20260923 23:14:41.155610 30862 raft_consensus.cc:348] T 00000000000000000000000000000000 P b02a236d7f0149b392339d6a556aa0c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b02a236d7f0149b392339d6a556aa0c3" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 44775 } }
I20260923 23:14:41.155756 30862 raft_consensus.cc:374] T 00000000000000000000000000000000 P b02a236d7f0149b392339d6a556aa0c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:14:41.155814 30862 raft_consensus.cc:729] T 00000000000000000000000000000000 P b02a236d7f0149b392339d6a556aa0c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b02a236d7f0149b392339d6a556aa0c3, State: Initialized, Role: FOLLOWER
I20260923 23:14:41.155988 30862 consensus_queue.cc:260] T 00000000000000000000000000000000 P b02a236d7f0149b392339d6a556aa0c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b02a236d7f0149b392339d6a556aa0c3" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 44775 } }
I20260923 23:14:41.156076 30862 raft_consensus.cc:388] T 00000000000000000000000000000000 P b02a236d7f0149b392339d6a556aa0c3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 23:14:41.156116 30862 raft_consensus.cc:482] T 00000000000000000000000000000000 P b02a236d7f0149b392339d6a556aa0c3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 23:14:41.156162 30862 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b02a236d7f0149b392339d6a556aa0c3 [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:14:41.156665 30862 raft_consensus.cc:504] T 00000000000000000000000000000000 P b02a236d7f0149b392339d6a556aa0c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b02a236d7f0149b392339d6a556aa0c3" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 44775 } }
I20260923 23:14:41.156786 30862 leader_election.cc:302] T 00000000000000000000000000000000 P b02a236d7f0149b392339d6a556aa0c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b02a236d7f0149b392339d6a556aa0c3; no voters:
I20260923 23:14:41.156944 30862 leader_election.cc:288] T 00000000000000000000000000000000 P b02a236d7f0149b392339d6a556aa0c3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 23:14:41.156971 30865 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b02a236d7f0149b392339d6a556aa0c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 23:14:41.157128 30865 raft_consensus.cc:686] T 00000000000000000000000000000000 P b02a236d7f0149b392339d6a556aa0c3 [term 1 LEADER]: Becoming Leader. State: Replica: b02a236d7f0149b392339d6a556aa0c3, State: Running, Role: LEADER
I20260923 23:14:41.157168 30862 sys_catalog.cc:564] T 00000000000000000000000000000000 P b02a236d7f0149b392339d6a556aa0c3 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 23:14:41.157218 30865 consensus_queue.cc:237] T 00000000000000000000000000000000 P b02a236d7f0149b392339d6a556aa0c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b02a236d7f0149b392339d6a556aa0c3" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 44775 } }
I20260923 23:14:41.157568 30866 sys_catalog.cc:455] T 00000000000000000000000000000000 P b02a236d7f0149b392339d6a556aa0c3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b02a236d7f0149b392339d6a556aa0c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b02a236d7f0149b392339d6a556aa0c3" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 44775 } } }
I20260923 23:14:41.157658 30866 sys_catalog.cc:458] T 00000000000000000000000000000000 P b02a236d7f0149b392339d6a556aa0c3 [sys.catalog]: This master's current role is: LEADER
I20260923 23:14:41.157716 30867 sys_catalog.cc:455] T 00000000000000000000000000000000 P b02a236d7f0149b392339d6a556aa0c3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b02a236d7f0149b392339d6a556aa0c3. Latest consensus state: current_term: 1 leader_uuid: "b02a236d7f0149b392339d6a556aa0c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b02a236d7f0149b392339d6a556aa0c3" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 44775 } } }
I20260923 23:14:41.157778 30867 sys_catalog.cc:458] T 00000000000000000000000000000000 P b02a236d7f0149b392339d6a556aa0c3 [sys.catalog]: This master's current role is: LEADER
I20260923 23:14:41.157871 30873 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 23:14:41.158357 30873 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 23:14:41.159235 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 30795
I20260923 23:14:41.159348 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790205259943518-25487-0/minicluster-data/master-0/wal/instance
I20260923 23:14:41.159498 25487 external_mini_cluster.cc:1428] Setting key 5b8cc09ca881b4ce8a668512e6b2c364
I20260923 23:14:41.160239 30873 catalog_manager.cc:1347] Generated new cluster ID: 7dde9b6e8d7c4c4ebf09faf3dcc6d82e
I20260923 23:14:41.160290 30873 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 23:14:41.165215 30885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:41.159943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55844 (local address 127.24.227.254:44775)
0923 23:14:41.160123 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:41.160132 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:41.160182 (+ 50us) server_negotiation.cc:407] Connection header received
0923 23:14:41.160289 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:41.160296 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:41.160357 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:41.160490 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:41.160726 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:41.162115 (+ 1389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:41.163162 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:41.163759 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:41.163920 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:41.164538 (+ 618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:41.164542 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:41.164543 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 23:14:41.164740 (+ 197us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:41.164778 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 23:14:41.164794 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:41.164866 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:41.164932 (+ 66us) server_negotiation.cc:299] Negotiation successful
0923 23:14:41.165010 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":82,"threads_started":1}
I20260923 23:14:41.166430 25487 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 23:14:41.169533 30873 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 23:14:41.170176 30873 catalog_manager.cc:1504] Loading token signing keys...
I20260923 23:14:41.183693 30873 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b02a236d7f0149b392339d6a556aa0c3: Generated new TSK 0
I20260923 23:14:41.183848 30873 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 23:14:41.196097 30885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:41.192632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55858 (local address 127.24.227.254:44775)
0923 23:14:41.192672 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:41.192674 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:41.192760 (+ 86us) server_negotiation.cc:407] Connection header received
0923 23:14:41.192798 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:41.192799 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:41.192809 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:41.192885 (+ 76us) server_negotiation.cc:226] Negotiated authn=JWT
0923 23:14:41.193003 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:41.193790 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:41.194528 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:41.194848 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:41.194916 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:41.195065 (+ 149us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0923 23:14:41.195914 (+ 849us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38599/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0923 23:14:41.196028 (+ 114us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:55858: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38599/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":23}
W20260923 23:14:41.196146 30885 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:55858: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38599/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260923 23:14:41.196395 30891 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 23:14:41.192546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:44775 (local address 127.0.0.1:55858)
0923 23:14:41.192687 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:41.192696 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:41.192757 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:41.192889 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:41.192889 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:41.192891 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0923 23:14:41.192936 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:41.192938 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:41.193860 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:41.193861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:41.194449 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:41.194450 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:41.194928 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:41.194928 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:41.194966 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:41.194970 (+ 4us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0923 23:14:41.196012 (+ 1042us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38599/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0923 23:14:41.196102 (+ 90us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.24.227.254:44775: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38599/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":97,"threads_started":1}
I20260923 23:14:41.196576 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 30795
E20260923 23:14:41.206646 30892 webserver.cc:533] Webserver: sslize failed:
E20260923 23:14:41.206697 30892 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-09-23T23:14:41Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (196 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-09-23T23:14:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T23:14:41Z Disabled control of system clock
I20260923 23:14:41.240641 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790205259943518-25487-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790205259943518-25487-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790205259943518-25487-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.254:39451
--webserver_interface=127.24.227.254
--webserver_port=0
--builtin_ntp_servers=127.24.227.212:37845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.227.254:39451
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790205259943518-25487-0/testchainca.cert with env {}
W20260923 23:14:41.317124 30896 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:14:41.317305 30896 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:14:41.317327 30896 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:14:41.318427 30896 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 23:14:41.318467 30896 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:14:41.318480 30896 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:14:41.318491 30896 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 23:14:41.318502 30896 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260923 23:14:41.318514 30896 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260923 23:14:41.318526 30896 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260923 23:14:41.319948 30896 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:37845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790205259943518-25487-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790205259943518-25487-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.227.254:39451
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.254:39451
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.227.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790205259943518-25487-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30896
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790205259943518-25487-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:14:41.320160 30896 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:14:41.320348 30896 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:14:41.325008 30901 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:41.325016 30904 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:41.325023 30902 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:14:41.325287 30896 server_base.cc:1034] running on GCE node
I20260923 23:14:41.325454 30896 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:14:41.325693 30896 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:14:41.326820 30896 hybrid_clock.cc:648] HybridClock initialized: now 1790205281326786 us; error 45 us; skew 500 ppm
I20260923 23:14:41.327982 30896 webserver.cc:460] Webserver started at http://127.24.227.254:32993/ using document root <none> and password file <none>
I20260923 23:14:41.328193 30896 fs_manager.cc:359] Metadata directory not provided
I20260923 23:14:41.328234 30896 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:14:41.328377 30896 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:14:41.329309 30896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790205259943518-25487-0/minicluster-data/master-0/data/instance:
uuid: "3b30585b6d194d8c83514aebf796432f"
format_stamp: "Formatted at 2026-09-23 23:14:41 on dist-test-slave-754n"
server_key: "c7cc799114f5af591f119ee95b7eb81a"
server_key_iv: "b5cfaa08f3018ad44766909123451e9e"
server_key_version: "encryptionkey@0"
I20260923 23:14:41.329668 30896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790205259943518-25487-0/minicluster-data/master-0/wal/instance:
uuid: "3b30585b6d194d8c83514aebf796432f"
format_stamp: "Formatted at 2026-09-23 23:14:41 on dist-test-slave-754n"
server_key: "c7cc799114f5af591f119ee95b7eb81a"
server_key_iv: "b5cfaa08f3018ad44766909123451e9e"
server_key_version: "encryptionkey@0"
I20260923 23:14:41.330860 30896 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 23:14:41.331521 30910 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:41.331683 30896 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 23:14:41.331784 30896 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790205259943518-25487-0/minicluster-data/master-0/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790205259943518-25487-0/minicluster-data/master-0/wal
uuid: "3b30585b6d194d8c83514aebf796432f"
format_stamp: "Formatted at 2026-09-23 23:14:41 on dist-test-slave-754n"
server_key: "c7cc799114f5af591f119ee95b7eb81a"
server_key_iv: "b5cfaa08f3018ad44766909123451e9e"
server_key_version: "encryptionkey@0"
I20260923 23:14:41.331858 30896 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790205259943518-25487-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790205259943518-25487-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790205259943518-25487-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:14:41.343834 30896 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:14:41.344120 30896 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:14:41.344280 30896 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:14:41.347697 30896 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.254:39451
I20260923 23:14:41.347728 30962 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.254:39451 every 8 connection(s)
I20260923 23:14:41.348071 30896 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
I20260923 23:14:41.348942 30963 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:14:41.350939 30963 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b30585b6d194d8c83514aebf796432f: Bootstrap starting.
I20260923 23:14:41.351497 30963 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3b30585b6d194d8c83514aebf796432f: Neither blocks nor log segments found. Creating new log.
I20260923 23:14:41.351765 30963 log.cc:824] T 00000000000000000000000000000000 P 3b30585b6d194d8c83514aebf796432f: Log is configured to *not* fsync() on all Append() calls
I20260923 23:14:41.352421 30963 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b30585b6d194d8c83514aebf796432f: No bootstrap required, opened a new log
I20260923 23:14:41.353919 30963 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3b30585b6d194d8c83514aebf796432f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b30585b6d194d8c83514aebf796432f" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 39451 } }
I20260923 23:14:41.354043 30963 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3b30585b6d194d8c83514aebf796432f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:14:41.354097 30963 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3b30585b6d194d8c83514aebf796432f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b30585b6d194d8c83514aebf796432f, State: Initialized, Role: FOLLOWER
I20260923 23:14:41.354292 30963 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3b30585b6d194d8c83514aebf796432f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b30585b6d194d8c83514aebf796432f" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 39451 } }
I20260923 23:14:41.354372 30963 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3b30585b6d194d8c83514aebf796432f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 23:14:41.354423 30963 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3b30585b6d194d8c83514aebf796432f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 23:14:41.354489 30963 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3b30585b6d194d8c83514aebf796432f [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:14:41.354794 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 30896
I20260923 23:14:41.354897 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790205259943518-25487-0/minicluster-data/master-0/wal/instance
I20260923 23:14:41.355085 25487 external_mini_cluster.cc:1428] Setting key ede653bb3edf8573353bb4c371549230
I20260923 23:14:41.355177 30963 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3b30585b6d194d8c83514aebf796432f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b30585b6d194d8c83514aebf796432f" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 39451 } }
I20260923 23:14:41.355276 30963 leader_election.cc:302] T 00000000000000000000000000000000 P 3b30585b6d194d8c83514aebf796432f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3b30585b6d194d8c83514aebf796432f; no voters:
I20260923 23:14:41.355422 30963 leader_election.cc:288] T 00000000000000000000000000000000 P 3b30585b6d194d8c83514aebf796432f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 23:14:41.355523 30966 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3b30585b6d194d8c83514aebf796432f [term 1 FOLLOWER]: Leader election won for term 1
I20260923 23:14:41.355662 30963 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3b30585b6d194d8c83514aebf796432f [sys.catalog]: configured and running, proceeding with master startup.
I20260923 23:14:41.355688 30966 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3b30585b6d194d8c83514aebf796432f [term 1 LEADER]: Becoming Leader. State: Replica: 3b30585b6d194d8c83514aebf796432f, State: Running, Role: LEADER
I20260923 23:14:41.355863 30966 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3b30585b6d194d8c83514aebf796432f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b30585b6d194d8c83514aebf796432f" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 39451 } }
I20260923 23:14:41.356490 30968 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b30585b6d194d8c83514aebf796432f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3b30585b6d194d8c83514aebf796432f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b30585b6d194d8c83514aebf796432f" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 39451 } } }
I20260923 23:14:41.356570 30968 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b30585b6d194d8c83514aebf796432f [sys.catalog]: This master's current role is: LEADER
I20260923 23:14:41.356724 30970 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b30585b6d194d8c83514aebf796432f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3b30585b6d194d8c83514aebf796432f. Latest consensus state: current_term: 1 leader_uuid: "3b30585b6d194d8c83514aebf796432f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b30585b6d194d8c83514aebf796432f" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 39451 } } }
I20260923 23:14:41.356775 30970 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b30585b6d194d8c83514aebf796432f [sys.catalog]: This master's current role is: LEADER
W20260923 23:14:41.357400 30983 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3b30585b6d194d8c83514aebf796432f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 23:14:41.357456 30983 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 23:14:41.357506 30978 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 23:14:41.357689 30978 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 23:14:41.359066 30978 catalog_manager.cc:1347] Generated new cluster ID: ba1f22a47cc54150862dbb239cb018c1
I20260923 23:14:41.359117 30978 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 23:14:41.361076 30969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:41.355519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35652 (local address 127.24.227.254:39451)
0923 23:14:41.355714 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:41.355722 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:41.355908 (+ 186us) server_negotiation.cc:407] Connection header received
0923 23:14:41.356054 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:41.356061 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:41.356133 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:41.356533 (+ 400us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:41.356634 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:41.357962 (+ 1328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:41.359065 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:41.359625 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:41.359745 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:41.360418 (+ 673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:41.360422 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:41.360422 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 23:14:41.360623 (+ 201us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:41.360660 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 23:14:41.360674 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:41.360744 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:41.360810 (+ 66us) server_negotiation.cc:299] Negotiation successful
0923 23:14:41.360883 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":64,"threads_started":1}
I20260923 23:14:41.364241 30978 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 23:14:41.364491 25487 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 23:14:41.364871 30978 catalog_manager.cc:1504] Loading token signing keys...
I20260923 23:14:41.369079 30978 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3b30585b6d194d8c83514aebf796432f: Generated new TSK 0
I20260923 23:14:41.369270 30978 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 23:14:41.436537 30969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:41.378130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35660 (local address 127.24.227.254:39451)
0923 23:14:41.378192 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:41.378193 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:41.378204 (+ 11us) server_negotiation.cc:407] Connection header received
0923 23:14:41.378286 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:41.378287 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:41.378296 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:41.378351 (+ 55us) server_negotiation.cc:226] Negotiated authn=JWT
0923 23:14:41.378492 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:41.380253 (+ 1761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:41.380975 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:41.381274 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:41.381341 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:41.381472 (+ 131us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0923 23:14:41.436273 (+ 54801us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0923 23:14:41.436462 (+ 189us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:35660: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":39}
W20260923 23:14:41.436614 30969 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:35660: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260923 23:14:41.436623 30992 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 23:14:41.378017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:39451 (local address 127.0.0.1:35660)
0923 23:14:41.378155 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:41.378166 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:41.378218 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:41.378369 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:41.378370 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:41.378372 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0923 23:14:41.378425 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:41.378426 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:41.380345 (+ 1919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:41.380346 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:41.380898 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:41.380899 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:41.381346 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:41.381347 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:41.381390 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:41.381394 (+ 4us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0923 23:14:41.436407 (+ 55013us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0923 23:14:41.436524 (+ 117us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.24.227.254:39451: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":89,"threads_started":1}
I20260923 23:14:41.437005 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 30896
2026-09-23T23:14:41Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (220 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (88 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (80 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 23:14:41 dist-test-slave-754n krb5kdc[31004](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 23:14:41 dist-test-slave-754n krb5kdc[31004](info): set up 2 sockets
Sep 23 23:14:41 dist-test-slave-754n krb5kdc[31004](info): commencing operation
krb5kdc: starting...
W20260923 23:14:42.591964 25487 mini_kdc.cc:121] Time spent starting KDC: real 0.969s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/krb5kdc/test-user.keytab.
Sep 23 23:14:42 dist-test-slave-754n krb5kdc[31004](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205282, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T23:14:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T23:14:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.227.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.227.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.227.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab.
Entry for principal kudu/127.24.227.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab.
I20260923 23:14:42.699795 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.254:41563
--webserver_interface=127.24.227.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab
--principal=kudu/127.24.227.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:38825
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.227.254:41563
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:14:42.774730 31017 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:14:42.774947 31017 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:14:42.774981 31017 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:14:42.776104 31017 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 23:14:42.776170 31017 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:14:42.776192 31017 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:14:42.776212 31017 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 23:14:42.776240 31017 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 23:14:42.777582 31017 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:38825
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.227.254:41563
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab
--principal=kudu/127.24.227.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.254:41563
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.227.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31017
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:14:42.777833 31017 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:14:42.778052 31017 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:14:42.782526 31025 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:14:42.782542 31017 server_base.cc:1034] running on GCE node
W20260923 23:14:42.782503 31023 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:42.782516 31022 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:14:42.783011 31017 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:14:42.783289 31017 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:14:42.784425 31017 hybrid_clock.cc:648] HybridClock initialized: now 1790205282784394 us; error 47 us; skew 500 ppm
Sep 23 23:14:42 dist-test-slave-754n krb5kdc[31004](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205282, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:14:42.786666 31017 init.cc:377] Logged in from keytab as kudu/127.24.227.254@KRBTEST.COM (short username kudu)
I20260923 23:14:42.787072 31017 webserver.cc:460] Webserver started at http://127.24.227.254:46699/ using document root <none> and password file <none>
I20260923 23:14:42.787243 31017 fs_manager.cc:359] Metadata directory not provided
I20260923 23:14:42.787279 31017 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:14:42.787359 31017 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:14:42.788367 31017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/minicluster-data/master-0/data/instance:
uuid: "86c1830c0d4c4fefa881153d541b8b39"
format_stamp: "Formatted at 2026-09-23 23:14:42 on dist-test-slave-754n"
server_key: "f3b7d0d3442d004a4c2b17acf8f54774"
server_key_iv: "aebf452f36785ecd42fa953d40a6e29c"
server_key_version: "encryptionkey@0"
I20260923 23:14:42.788647 31017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/minicluster-data/master-0/wal/instance:
uuid: "86c1830c0d4c4fefa881153d541b8b39"
format_stamp: "Formatted at 2026-09-23 23:14:42 on dist-test-slave-754n"
server_key: "f3b7d0d3442d004a4c2b17acf8f54774"
server_key_iv: "aebf452f36785ecd42fa953d40a6e29c"
server_key_version: "encryptionkey@0"
I20260923 23:14:42.789858 31017 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 23:14:42.790524 31032 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:42.790729 31017 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 23:14:42.790797 31017 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/minicluster-data/master-0/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/minicluster-data/master-0/wal
uuid: "86c1830c0d4c4fefa881153d541b8b39"
format_stamp: "Formatted at 2026-09-23 23:14:42 on dist-test-slave-754n"
server_key: "f3b7d0d3442d004a4c2b17acf8f54774"
server_key_iv: "aebf452f36785ecd42fa953d40a6e29c"
server_key_version: "encryptionkey@0"
I20260923 23:14:42.790884 31017 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:14:42.799186 31017 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:14:42.801074 31017 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:14:42.801251 31017 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:14:42.805510 31017 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.254:41563
I20260923 23:14:42.805552 31084 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.254:41563 every 8 connection(s)
I20260923 23:14:42.805828 31017 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
I20260923 23:14:42.806610 31085 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:14:42.808638 31085 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86c1830c0d4c4fefa881153d541b8b39: Bootstrap starting.
I20260923 23:14:42.809110 31085 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 86c1830c0d4c4fefa881153d541b8b39: Neither blocks nor log segments found. Creating new log.
I20260923 23:14:42.809336 31085 log.cc:824] T 00000000000000000000000000000000 P 86c1830c0d4c4fefa881153d541b8b39: Log is configured to *not* fsync() on all Append() calls
I20260923 23:14:42.809942 31085 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86c1830c0d4c4fefa881153d541b8b39: No bootstrap required, opened a new log
I20260923 23:14:42.811429 31085 raft_consensus.cc:348] T 00000000000000000000000000000000 P 86c1830c0d4c4fefa881153d541b8b39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86c1830c0d4c4fefa881153d541b8b39" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41563 } }
I20260923 23:14:42.811561 31085 raft_consensus.cc:374] T 00000000000000000000000000000000 P 86c1830c0d4c4fefa881153d541b8b39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:14:42.811581 31085 raft_consensus.cc:729] T 00000000000000000000000000000000 P 86c1830c0d4c4fefa881153d541b8b39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86c1830c0d4c4fefa881153d541b8b39, State: Initialized, Role: FOLLOWER
I20260923 23:14:42.811666 31085 consensus_queue.cc:260] T 00000000000000000000000000000000 P 86c1830c0d4c4fefa881153d541b8b39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86c1830c0d4c4fefa881153d541b8b39" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41563 } }
I20260923 23:14:42.811760 31085 raft_consensus.cc:388] T 00000000000000000000000000000000 P 86c1830c0d4c4fefa881153d541b8b39 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 23:14:42.811793 31085 raft_consensus.cc:482] T 00000000000000000000000000000000 P 86c1830c0d4c4fefa881153d541b8b39 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 23:14:42.811867 31085 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 86c1830c0d4c4fefa881153d541b8b39 [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:14:42.812390 31085 raft_consensus.cc:504] T 00000000000000000000000000000000 P 86c1830c0d4c4fefa881153d541b8b39 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86c1830c0d4c4fefa881153d541b8b39" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41563 } }
I20260923 23:14:42.812502 31085 leader_election.cc:302] T 00000000000000000000000000000000 P 86c1830c0d4c4fefa881153d541b8b39 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 86c1830c0d4c4fefa881153d541b8b39; no voters:
I20260923 23:14:42.812675 31085 leader_election.cc:288] T 00000000000000000000000000000000 P 86c1830c0d4c4fefa881153d541b8b39 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 23:14:42.812718 31088 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 86c1830c0d4c4fefa881153d541b8b39 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 23:14:42.812906 31085 sys_catalog.cc:564] T 00000000000000000000000000000000 P 86c1830c0d4c4fefa881153d541b8b39 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 23:14:42.813054 31088 raft_consensus.cc:686] T 00000000000000000000000000000000 P 86c1830c0d4c4fefa881153d541b8b39 [term 1 LEADER]: Becoming Leader. State: Replica: 86c1830c0d4c4fefa881153d541b8b39, State: Running, Role: LEADER
I20260923 23:14:42.813162 31088 consensus_queue.cc:237] T 00000000000000000000000000000000 P 86c1830c0d4c4fefa881153d541b8b39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86c1830c0d4c4fefa881153d541b8b39" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41563 } }
I20260923 23:14:42.813527 31090 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86c1830c0d4c4fefa881153d541b8b39 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 86c1830c0d4c4fefa881153d541b8b39. Latest consensus state: current_term: 1 leader_uuid: "86c1830c0d4c4fefa881153d541b8b39" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86c1830c0d4c4fefa881153d541b8b39" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41563 } } }
I20260923 23:14:42.813551 31089 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86c1830c0d4c4fefa881153d541b8b39 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "86c1830c0d4c4fefa881153d541b8b39" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86c1830c0d4c4fefa881153d541b8b39" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41563 } } }
I20260923 23:14:42.813608 31090 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86c1830c0d4c4fefa881153d541b8b39 [sys.catalog]: This master's current role is: LEADER
I20260923 23:14:42.813627 31089 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86c1830c0d4c4fefa881153d541b8b39 [sys.catalog]: This master's current role is: LEADER
I20260923 23:14:42.813629 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 31017
I20260923 23:14:42.813719 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/minicluster-data/master-0/wal/instance
I20260923 23:14:42.813908 25487 external_mini_cluster.cc:1428] Setting key d99dfaf96e072a6066013d86d2df6d5e
I20260923 23:14:42.813902 31095 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 23:14:42.814484 31095 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 23:14:42.816007 31095 catalog_manager.cc:1347] Generated new cluster ID: 831d793b76c441f8b797e507b0436704
I20260923 23:14:42.816054 31095 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 23:14:42 dist-test-slave-754n krb5kdc[31004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205282, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
I20260923 23:14:42.830457 31099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:42.814312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48262 (local address 127.24.227.254:41563)
0923 23:14:42.820002 (+ 5690us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:42.820010 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:42.820037 (+ 27us) server_negotiation.cc:407] Connection header received
0923 23:14:42.820127 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:42.820131 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:42.820168 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:42.820267 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:42.820962 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:42.822357 (+ 1395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:42.823405 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:42.824146 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:42.824262 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:42.826945 (+ 2683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:42.826949 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:42.826949 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:42.826959 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:42.828742 (+ 1783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:42.829241 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:42.829244 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:42.829244 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:42.829350 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:42.829579 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:42.829580 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:42.829580 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:42.829847 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:42.829995 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:42.830109 (+ 114us) server_negotiation.cc:299] Negotiation successful
0923 23:14:42.830234 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1623,"thread_start_us":98,"threads_started":1}
I20260923 23:14:42.830878 31095 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 23:14:42.831342 31095 catalog_manager.cc:1504] Loading token signing keys...
I20260923 23:14:42.839931 31095 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 86c1830c0d4c4fefa881153d541b8b39: Generated new TSK 0
I20260923 23:14:42.840082 31095 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.227.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.227.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.227.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.193.keytab.
Entry for principal kudu/127.24.227.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.193.keytab.
I20260923 23:14:42.864408 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.227.193:0
--local_ip_for_outbound_sockets=127.24.227.193
--webserver_interface=127.24.227.193
--webserver_port=0
--tserver_master_addrs=127.24.227.254:41563
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.193.keytab
--principal=kudu/127.24.227.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:38825
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:14:42.944553 31111 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:14:42.944748 31111 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:14:42.944770 31111 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:14:42.945904 31111 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:14:42.945952 31111 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:14:42.946007 31111 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.227.193
I20260923 23:14:42.947438 31111 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:38825
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.193.keytab
--principal=kudu/127.24.227.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.227.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.227.193
--webserver_port=0
--tserver_master_addrs=127.24.227.254:41563
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31111
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.227.193
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:14:42.947665 31111 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:14:42.947952 31111 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:14:42.952347 31116 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:42.952358 31117 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:42.952391 31119 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:14:42.952620 31111 server_base.cc:1034] running on GCE node
I20260923 23:14:42.952833 31111 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:14:42.953066 31111 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:14:42.954221 31111 hybrid_clock.cc:648] HybridClock initialized: now 1790205282954209 us; error 24 us; skew 500 ppm
Sep 23 23:14:42 dist-test-slave-754n krb5kdc[31004](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205282, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:14:42.956565 31111 init.cc:377] Logged in from keytab as kudu/127.24.227.193@KRBTEST.COM (short username kudu)
I20260923 23:14:42.957017 31111 webserver.cc:460] Webserver started at http://127.24.227.193:46271/ using document root <none> and password file <none>
I20260923 23:14:42.957222 31111 fs_manager.cc:359] Metadata directory not provided
I20260923 23:14:42.957294 31111 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:14:42.957436 31111 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:14:42.958446 31111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/minicluster-data/ts-0/data/instance:
uuid: "e74d4b103f0a4ccab29fe15844fcaf85"
format_stamp: "Formatted at 2026-09-23 23:14:42 on dist-test-slave-754n"
server_key: "bb0395d6358a557aeb32d7c8bba1ecdc"
server_key_iv: "3a2d33b8543c5ac1363665a5e78ef8f5"
server_key_version: "encryptionkey@0"
I20260923 23:14:42.958796 31111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/minicluster-data/ts-0/wal/instance:
uuid: "e74d4b103f0a4ccab29fe15844fcaf85"
format_stamp: "Formatted at 2026-09-23 23:14:42 on dist-test-slave-754n"
server_key: "bb0395d6358a557aeb32d7c8bba1ecdc"
server_key_iv: "3a2d33b8543c5ac1363665a5e78ef8f5"
server_key_version: "encryptionkey@0"
I20260923 23:14:42.960057 31111 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 23:14:42.960713 31126 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:42.960880 31111 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 23:14:42.960969 31111 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/minicluster-data/ts-0/wal
uuid: "e74d4b103f0a4ccab29fe15844fcaf85"
format_stamp: "Formatted at 2026-09-23 23:14:42 on dist-test-slave-754n"
server_key: "bb0395d6358a557aeb32d7c8bba1ecdc"
server_key_iv: "3a2d33b8543c5ac1363665a5e78ef8f5"
server_key_version: "encryptionkey@0"
I20260923 23:14:42.961043 31111 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:14:42.985455 31111 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:14:42.987195 31111 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:14:42.987344 31111 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:14:42.987533 31111 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 23:14:42.987813 31111 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 23:14:42.987864 31111 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:42.987932 31111 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 23:14:42.987958 31111 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:42.993584 31111 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.193:43231
I20260923 23:14:42.993629 31239 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.193:43231 every 8 connection(s)
I20260923 23:14:42.993909 31111 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/minicluster-data/ts-0/data/info.pb
I20260923 23:14:42.998476 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 31111
I20260923 23:14:42.998582 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/minicluster-data/ts-0/wal/instance
I20260923 23:14:42.998762 25487 external_mini_cluster.cc:1428] Setting key 9129bffc1fa07f50c118fde2918bc6f6
Sep 23 23:14:42 dist-test-slave-754n krb5kdc[31004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205282, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.193@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
I20260923 23:14:43.003187 31099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:42.994514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.193:33495 (local address 127.24.227.254:41563)
0923 23:14:42.994572 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:42.994574 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:42.994811 (+ 237us) server_negotiation.cc:407] Connection header received
0923 23:14:42.995635 (+ 824us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:42.995636 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:42.995648 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:42.995695 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:42.996323 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:42.996789 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:42.997663 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:42.997845 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:42.997893 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:43.000197 (+ 2304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:43.000198 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:43.000198 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:43.000205 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:43.001790 (+ 1585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:43.002208 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:43.002209 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:43.002210 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:43.002273 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:43.002493 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:43.002494 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:43.002494 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:43.002634 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:43.002705 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:43.002928 (+ 223us) server_negotiation.cc:299] Negotiation successful
0923 23:14:43.003030 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260923 23:14:43.003207 31242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:42.994528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41563 (local address 127.24.227.193:33495)
0923 23:14:42.994680 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:42.994711 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:42.995550 (+ 839us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:42.995762 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:42.995765 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:42.996084 (+ 319us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:14:42.996250 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:42.996254 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:42.996860 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:42.996861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:42.997591 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:42.997593 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:42.997912 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:42.997912 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:42.998001 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:42.998408 (+ 407us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:14:42.998418 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:14:43.000109 (+ 1691us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:14:43.001867 (+ 1758us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:43.001899 (+ 32us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:43.001900 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:43.002133 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:43.002339 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:43.002340 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:43.002340 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:43.002428 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:43.002699 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:14:43.002703 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:14:43.002800 (+ 97us) client_negotiation.cc:769] Sending connection context
0923 23:14:43.002875 (+ 75us) client_negotiation.cc:240] Negotiation successful
0923 23:14:43.003056 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":79,"threads_started":1}
I20260923 23:14:43.003693 31240 heartbeater.cc:347] Connected to a master server at 127.24.227.254:41563
I20260923 23:14:43.003789 31240 heartbeater.cc:464] Registering TS with master...
I20260923 23:14:43.004002 31240 heartbeater.cc:511] Master 127.24.227.254:41563 requested a full tablet report, sending...
I20260923 23:14:43.004501 31049 ts_manager.cc:194] Registered new tserver with Master: e74d4b103f0a4ccab29fe15844fcaf85 (127.24.227.193:43231)
I20260923 23:14:43.005107 31049 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.227.193@KRBTEST.COM'} at 127.24.227.193:33495
WARNING: no policy specified for kudu/127.24.227.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.227.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.227.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.194.keytab.
Entry for principal kudu/127.24.227.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.194.keytab.
I20260923 23:14:43.022161 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.227.194:0
--local_ip_for_outbound_sockets=127.24.227.194
--webserver_interface=127.24.227.194
--webserver_port=0
--tserver_master_addrs=127.24.227.254:41563
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.194.keytab
--principal=kudu/127.24.227.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:38825
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:14:43.096246 31245 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:14:43.096448 31245 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:14:43.096493 31245 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:14:43.097538 31245 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:14:43.097599 31245 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:14:43.097638 31245 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.227.194
I20260923 23:14:43.099030 31245 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:38825
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.194.keytab
--principal=kudu/127.24.227.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.227.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.227.194
--webserver_port=0
--tserver_master_addrs=127.24.227.254:41563
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31245
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.227.194
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:14:43.099267 31245 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:14:43.099476 31245 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:14:43.104092 31250 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:43.104075 31253 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:43.104094 31251 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:14:43.104120 31245 server_base.cc:1034] running on GCE node
I20260923 23:14:43.104470 31245 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:14:43.104707 31245 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:14:43.105860 31245 hybrid_clock.cc:648] HybridClock initialized: now 1790205283105849 us; error 26 us; skew 500 ppm
Sep 23 23:14:43 dist-test-slave-754n krb5kdc[31004](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205283, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:14:43.108273 31245 init.cc:377] Logged in from keytab as kudu/127.24.227.194@KRBTEST.COM (short username kudu)
I20260923 23:14:43.108611 31245 webserver.cc:460] Webserver started at http://127.24.227.194:39587/ using document root <none> and password file <none>
I20260923 23:14:43.108772 31245 fs_manager.cc:359] Metadata directory not provided
I20260923 23:14:43.108808 31245 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:14:43.108898 31245 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:14:43.109760 31245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/minicluster-data/ts-1/data/instance:
uuid: "341990e886874350ace11ea10fa3065c"
format_stamp: "Formatted at 2026-09-23 23:14:43 on dist-test-slave-754n"
server_key: "93ece72f4ddfce19a2bbd99592dbbdcf"
server_key_iv: "140fde45b71bbaf28b4215dd9108c930"
server_key_version: "encryptionkey@0"
I20260923 23:14:43.110047 31245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/minicluster-data/ts-1/wal/instance:
uuid: "341990e886874350ace11ea10fa3065c"
format_stamp: "Formatted at 2026-09-23 23:14:43 on dist-test-slave-754n"
server_key: "93ece72f4ddfce19a2bbd99592dbbdcf"
server_key_iv: "140fde45b71bbaf28b4215dd9108c930"
server_key_version: "encryptionkey@0"
I20260923 23:14:43.111187 31245 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 23:14:43.111740 31260 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:43.111943 31245 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:43.112038 31245 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/minicluster-data/ts-1/wal
uuid: "341990e886874350ace11ea10fa3065c"
format_stamp: "Formatted at 2026-09-23 23:14:43 on dist-test-slave-754n"
server_key: "93ece72f4ddfce19a2bbd99592dbbdcf"
server_key_iv: "140fde45b71bbaf28b4215dd9108c930"
server_key_version: "encryptionkey@0"
I20260923 23:14:43.112116 31245 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:14:43.129616 31245 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:14:43.131795 31245 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:14:43.132000 31245 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:14:43.132227 31245 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 23:14:43.132521 31245 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 23:14:43.132577 31245 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:43.132609 31245 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 23:14:43.132635 31245 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:43.138191 31245 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.194:43625
I20260923 23:14:43.138257 31373 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.194:43625 every 8 connection(s)
I20260923 23:14:43.138566 31245 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/minicluster-data/ts-1/data/info.pb
Sep 23 23:14:43 dist-test-slave-754n krb5kdc[31004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205283, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.194@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
I20260923 23:14:43.146401 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 31245
I20260923 23:14:43.146528 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/minicluster-data/ts-1/wal/instance
I20260923 23:14:43.146746 25487 external_mini_cluster.cc:1428] Setting key b9c6cd0567f5e4338891f3bfb8f197e5
I20260923 23:14:43.147367 31099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:43.139245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.194:59639 (local address 127.24.227.254:41563)
0923 23:14:43.139289 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:43.139291 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:43.139468 (+ 177us) server_negotiation.cc:407] Connection header received
0923 23:14:43.140400 (+ 932us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:43.140401 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:43.140410 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:43.140468 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:43.141009 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:43.141477 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:43.142511 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:43.142698 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:43.142754 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:43.144699 (+ 1945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:43.144700 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:43.144701 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:43.144709 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:43.145899 (+ 1190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:43.146342 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:43.146343 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:43.146343 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:43.146404 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:43.146686 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:43.146687 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:43.146687 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:43.146824 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:43.146906 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:43.147186 (+ 280us) server_negotiation.cc:299] Negotiation successful
0923 23:14:43.147280 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 23:14:43.147511 31376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:43.139193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41563 (local address 127.24.227.194:59639)
0923 23:14:43.139342 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:43.139380 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:43.140314 (+ 934us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:43.140542 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:43.140545 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:43.140767 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:14:43.140933 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:43.140937 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:43.141547 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:43.141548 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:43.142430 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:43.142432 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:43.142769 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:43.142770 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:43.142862 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:43.143256 (+ 394us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:14:43.143264 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:14:43.144599 (+ 1335us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:14:43.146012 (+ 1413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:43.146018 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:43.146020 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:43.146259 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:43.146480 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:43.146481 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:43.146482 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:43.146618 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:43.146910 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:14:43.146916 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:14:43.147047 (+ 131us) client_negotiation.cc:769] Sending connection context
0923 23:14:43.147146 (+ 99us) client_negotiation.cc:240] Negotiation successful
0923 23:14:43.147292 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":88,"threads_started":1}
I20260923 23:14:43.147959 31374 heartbeater.cc:347] Connected to a master server at 127.24.227.254:41563
I20260923 23:14:43.148056 31374 heartbeater.cc:464] Registering TS with master...
I20260923 23:14:43.148229 31374 heartbeater.cc:511] Master 127.24.227.254:41563 requested a full tablet report, sending...
I20260923 23:14:43.148573 31049 ts_manager.cc:194] Registered new tserver with Master: 341990e886874350ace11ea10fa3065c (127.24.227.194:43625)
I20260923 23:14:43.149019 31049 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.227.194@KRBTEST.COM'} at 127.24.227.194:59639
WARNING: no policy specified for kudu/127.24.227.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.227.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.227.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.195.keytab.
Entry for principal kudu/127.24.227.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.195.keytab.
I20260923 23:14:43.168587 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.227.195:0
--local_ip_for_outbound_sockets=127.24.227.195
--webserver_interface=127.24.227.195
--webserver_port=0
--tserver_master_addrs=127.24.227.254:41563
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.195.keytab
--principal=kudu/127.24.227.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:38825
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:14:43.244840 31379 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:14:43.245097 31379 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:14:43.245132 31379 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:14:43.246271 31379 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:14:43.246333 31379 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:14:43.246374 31379 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.227.195
I20260923 23:14:43.247766 31379 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:38825
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.195.keytab
--principal=kudu/127.24.227.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.227.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.227.195
--webserver_port=0
--tserver_master_addrs=127.24.227.254:41563
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31379
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.227.195
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:14:43.248035 31379 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:14:43.248265 31379 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:14:43.252650 31384 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:14:43.252764 31379 server_base.cc:1034] running on GCE node
W20260923 23:14:43.252629 31387 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:43.252619 31385 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:14:43.253051 31379 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:14:43.253244 31379 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:14:43.254379 31379 hybrid_clock.cc:648] HybridClock initialized: now 1790205283254367 us; error 27 us; skew 500 ppm
Sep 23 23:14:43 dist-test-slave-754n krb5kdc[31004](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205283, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:14:43.256680 31379 init.cc:377] Logged in from keytab as kudu/127.24.227.195@KRBTEST.COM (short username kudu)
I20260923 23:14:43.257105 31379 webserver.cc:460] Webserver started at http://127.24.227.195:40691/ using document root <none> and password file <none>
I20260923 23:14:43.257275 31379 fs_manager.cc:359] Metadata directory not provided
I20260923 23:14:43.257313 31379 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:14:43.257392 31379 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:14:43.258282 31379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/minicluster-data/ts-2/data/instance:
uuid: "39edd7f495f041b8b0400be574cb3242"
format_stamp: "Formatted at 2026-09-23 23:14:43 on dist-test-slave-754n"
server_key: "16c515070b24da30cdeeeb8cbe5854dc"
server_key_iv: "31ef53f6670ee25c82d61b1869997741"
server_key_version: "encryptionkey@0"
I20260923 23:14:43.258567 31379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/minicluster-data/ts-2/wal/instance:
uuid: "39edd7f495f041b8b0400be574cb3242"
format_stamp: "Formatted at 2026-09-23 23:14:43 on dist-test-slave-754n"
server_key: "16c515070b24da30cdeeeb8cbe5854dc"
server_key_iv: "31ef53f6670ee25c82d61b1869997741"
server_key_version: "encryptionkey@0"
I20260923 23:14:43.259706 31379 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260923 23:14:43.260365 31394 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:43.260499 31379 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:43.260576 31379 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/minicluster-data/ts-2/wal
uuid: "39edd7f495f041b8b0400be574cb3242"
format_stamp: "Formatted at 2026-09-23 23:14:43 on dist-test-slave-754n"
server_key: "16c515070b24da30cdeeeb8cbe5854dc"
server_key_iv: "31ef53f6670ee25c82d61b1869997741"
server_key_version: "encryptionkey@0"
I20260923 23:14:43.260623 31379 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:14:43.279958 31379 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:14:43.281541 31379 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:14:43.281651 31379 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:14:43.281791 31379 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 23:14:43.282086 31379 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 23:14:43.282117 31379 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:43.282167 31379 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 23:14:43.282199 31379 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:43.287521 31379 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.195:40575
I20260923 23:14:43.287575 31507 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.195:40575 every 8 connection(s)
I20260923 23:14:43.287864 31379 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/minicluster-data/ts-2/data/info.pb
I20260923 23:14:43.292780 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 31379
I20260923 23:14:43.292898 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/minicluster-data/ts-2/wal/instance
I20260923 23:14:43.293071 25487 external_mini_cluster.cc:1428] Setting key 3cef3f2d210ef01ae7c4c1a694727ef6
Sep 23 23:14:43 dist-test-slave-754n krb5kdc[31004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205283, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.195@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
I20260923 23:14:43.296772 31099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:43.288529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.195:59793 (local address 127.24.227.254:41563)
0923 23:14:43.288576 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:43.288578 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:43.288775 (+ 197us) server_negotiation.cc:407] Connection header received
0923 23:14:43.289700 (+ 925us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:43.289702 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:43.289715 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:43.289768 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:43.290336 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:43.290840 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:43.291779 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:43.292040 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:43.292121 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:43.294375 (+ 2254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:43.294376 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:43.294376 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:43.294383 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:43.295471 (+ 1088us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:43.295850 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:43.295852 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:43.295852 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:43.295922 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:43.296176 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:43.296177 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:43.296177 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:43.296289 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:43.296362 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:43.296615 (+ 253us) server_negotiation.cc:299] Negotiation successful
0923 23:14:43.296689 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260923 23:14:43.296932 31510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:43.288507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41563 (local address 127.24.227.195:59793)
0923 23:14:43.288657 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:43.288688 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:43.289562 (+ 874us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:43.289838 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:43.289842 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:43.290079 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:14:43.290242 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:43.290246 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:43.290932 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:43.290933 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:43.291690 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:43.291692 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:43.292130 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:43.292131 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:43.292252 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:43.292704 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:14:43.292711 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:14:43.294281 (+ 1570us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:14:43.295552 (+ 1271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:43.295559 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:43.295561 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:43.295762 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:43.295993 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:43.295994 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:43.295995 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:43.296109 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:43.296354 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:14:43.296359 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:14:43.296485 (+ 126us) client_negotiation.cc:769] Sending connection context
0923 23:14:43.296592 (+ 107us) client_negotiation.cc:240] Negotiation successful
0923 23:14:43.296726 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":77,"threads_started":1}
I20260923 23:14:43.297330 31508 heartbeater.cc:347] Connected to a master server at 127.24.227.254:41563
I20260923 23:14:43.297428 31508 heartbeater.cc:464] Registering TS with master...
I20260923 23:14:43.297591 31508 heartbeater.cc:511] Master 127.24.227.254:41563 requested a full tablet report, sending...
I20260923 23:14:43.297912 31049 ts_manager.cc:194] Registered new tserver with Master: 39edd7f495f041b8b0400be574cb3242 (127.24.227.195:40575)
I20260923 23:14:43.298215 31049 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.227.195@KRBTEST.COM'} at 127.24.227.195:59793
I20260923 23:14:43.304638 25487 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/krb5kdc/test-admin.keytab.
Sep 23 23:14:43 dist-test-slave-754n krb5kdc[31004](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205283, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:14:43.319278 25487 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260923 23:14:43.320667 31515 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 23:14:43.319986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51384 (local address 127.24.227.193:43231)
0923 23:14:43.320149 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:43.320158 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:43.320167 (+ 9us) server_negotiation.cc:407] Connection header received
0923 23:14:43.320188 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:43.320189 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:43.320201 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:43.320261 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:43.320571 (+ 310us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51384: BlockingRecv error: recv got EOF from 127.0.0.1:51384 (error 108)
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":79,"threads_started":1}
W20260923 23:14:43.320825 25487 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/krb5kdc/krb5cc' of type FILE
Sep 23 23:14:43 dist-test-slave-754n krb5kdc[31004](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205283, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:14:43.321669 25487 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 23 23:14:43 dist-test-slave-754n krb5kdc[31004](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205283, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:14:43.322506 25487 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 23 23:14:43 dist-test-slave-754n krb5kdc[31004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205283, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.227.193@KRBTEST.COM
I20260923 23:14:43.329692 31515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:43.323018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51400 (local address 127.24.227.193:43231)
0923 23:14:43.323058 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:43.323059 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:43.323114 (+ 55us) server_negotiation.cc:407] Connection header received
0923 23:14:43.323165 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:43.323166 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:43.323174 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:43.323219 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:43.323557 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:43.324334 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:43.325004 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:43.325293 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:43.325343 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:43.327230 (+ 1887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:43.327238 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:43.327238 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:43.327252 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:43.328628 (+ 1376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:43.328956 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:43.328958 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:43.328959 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:43.329011 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:43.329195 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:43.329196 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:43.329196 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:43.329316 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:43.329461 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:43.329534 (+ 73us) server_negotiation.cc:299] Negotiation successful
0923 23:14:43.329598 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
W20260923 23:14:43.331254 31515 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 23:14:43.330784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51404 (local address 127.24.227.193:43231)
0923 23:14:43.330818 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:43.330820 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:43.330857 (+ 37us) server_negotiation.cc:407] Connection header received
0923 23:14:43.330902 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:43.330903 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:43.330911 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:43.330952 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:43.331207 (+ 255us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51404: BlockingRecv error: recv got EOF from 127.0.0.1:51404 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260923 23:14:43.331351 25487 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/krb5kdc/krb5cc' of type FILE
Sep 23 23:14:43 dist-test-slave-754n krb5kdc[31004](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205283, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:14:43.332242 25487 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 23 23:14:43 dist-test-slave-754n krb5kdc[31004](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205283, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:14:43.333025 25487 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 23 23:14:43 dist-test-slave-754n krb5kdc[31004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205283, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.227.193@KRBTEST.COM
I20260923 23:14:43.339125 31515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:43.333438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51412 (local address 127.24.227.193:43231)
0923 23:14:43.333492 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:43.333493 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:43.333558 (+ 65us) server_negotiation.cc:407] Connection header received
0923 23:14:43.333591 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:43.333592 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:43.333599 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:43.333636 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:43.333952 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:43.334398 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:43.335002 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:43.335182 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:43.335249 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:43.337014 (+ 1765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:43.337015 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:43.337015 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:43.337020 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:43.338265 (+ 1245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:43.338531 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:43.338531 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:43.338532 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:43.338577 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:43.338720 (+ 143us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:43.338720 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:43.338720 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:43.338788 (+ 68us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:43.338845 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:43.338928 (+ 83us) server_negotiation.cc:299] Negotiation successful
0923 23:14:43.338979 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
W20260923 23:14:43.340657 31515 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 23:14:43.340126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51416 (local address 127.24.227.193:43231)
0923 23:14:43.340164 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:43.340166 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:43.340216 (+ 50us) server_negotiation.cc:407] Connection header received
0923 23:14:43.340305 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:43.340305 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:43.340312 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:43.340348 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:43.340620 (+ 272us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51416: BlockingRecv error: recv got EOF from 127.0.0.1:51416 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260923 23:14:43.340714 25487 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790205259943518-25487-0/krb5kdc/krb5cc' of type FILE
Sep 23 23:14:43 dist-test-slave-754n krb5kdc[31004](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205283, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:14:43.341570 25487 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 23 23:14:43 dist-test-slave-754n krb5kdc[31004](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205283, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:14:43.342490 25487 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 23 23:14:43 dist-test-slave-754n krb5kdc[31004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205283, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.227.193@KRBTEST.COM
I20260923 23:14:43.348457 31515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:43.342910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51432 (local address 127.24.227.193:43231)
0923 23:14:43.342943 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:43.342945 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:43.343012 (+ 67us) server_negotiation.cc:407] Connection header received
0923 23:14:43.343058 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:43.343059 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:43.343065 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:43.343106 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:43.343423 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:43.343833 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:43.344482 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:43.344652 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:43.344699 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:43.346478 (+ 1779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:43.346479 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:43.346479 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:43.346485 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:43.347587 (+ 1102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:43.347905 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:43.347906 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:43.347906 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:43.347966 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:43.348133 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:43.348133 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:43.348133 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:43.348204 (+ 71us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:43.348275 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:43.348364 (+ 89us) server_negotiation.cc:299] Negotiation successful
0923 23:14:43.348411 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260923 23:14:43.349035 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 31111
I20260923 23:14:43.353598 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 31245
I20260923 23:14:43.358054 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 31379
I20260923 23:14:43.362145 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 31017
2026-09-23T23:14:43Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (1758 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 23:14:43 dist-test-slave-754n krb5kdc[31528](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 23:14:43 dist-test-slave-754n krb5kdc[31528](info): set up 2 sockets
Sep 23 23:14:43 dist-test-slave-754n krb5kdc[31528](info): commencing operation
krb5kdc: starting...
W20260923 23:14:44.360074 25487 mini_kdc.cc:121] Time spent starting KDC: real 0.979s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/krb5kdc/test-user.keytab.
Sep 23 23:14:44 dist-test-slave-754n krb5kdc[31528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205284, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T23:14:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T23:14:44Z Disabled control of system clock
WARNING: no policy specified for oryx/127.24.227.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.227.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.227.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/krb5kdc/oryx_127.24.227.254.keytab.
Entry for principal oryx/127.24.227.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/krb5kdc/oryx_127.24.227.254.keytab.
I20260923 23:14:44.464885 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.254:41951
--webserver_interface=127.24.227.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/krb5kdc/oryx_127.24.227.254.keytab
--principal=oryx/127.24.227.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:42907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.227.254:41951
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:14:44.540673 31541 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:14:44.540853 31541 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:14:44.540874 31541 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:14:44.541983 31541 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 23:14:44.542024 31541 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:14:44.542038 31541 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:14:44.542078 31541 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 23:14:44.542107 31541 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 23:14:44.543542 31541 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:42907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.227.254:41951
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/krb5kdc/oryx_127.24.227.254.keytab
--principal=oryx/127.24.227.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.254:41951
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.227.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31541
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:14:44.543772 31541 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:14:44.544032 31541 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:14:44.548565 31547 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:44.548564 31546 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:14:44.548635 31541 server_base.cc:1034] running on GCE node
W20260923 23:14:44.548568 31549 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:14:44.548951 31541 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:14:44.549206 31541 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:14:44.550340 31541 hybrid_clock.cc:648] HybridClock initialized: now 1790205284550326 us; error 29 us; skew 500 ppm
Sep 23 23:14:44 dist-test-slave-754n krb5kdc[31528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205284, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.227.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:14:44.552628 31541 init.cc:377] Logged in from keytab as oryx/127.24.227.254@KRBTEST.COM (short username oryx)
I20260923 23:14:44.553020 31541 webserver.cc:460] Webserver started at http://127.24.227.254:44811/ using document root <none> and password file <none>
I20260923 23:14:44.553224 31541 fs_manager.cc:359] Metadata directory not provided
I20260923 23:14:44.553305 31541 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:14:44.553450 31541 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:14:44.554440 31541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/master-0/data/instance:
uuid: "2ddae489e6844395a47c75343be04fa5"
format_stamp: "Formatted at 2026-09-23 23:14:44 on dist-test-slave-754n"
server_key: "2edc19541e9f7b43b026c09ff104e0d3"
server_key_iv: "d47de327ce4b223dde70f6b671140694"
server_key_version: "encryptionkey@0"
I20260923 23:14:44.554791 31541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/master-0/wal/instance:
uuid: "2ddae489e6844395a47c75343be04fa5"
format_stamp: "Formatted at 2026-09-23 23:14:44 on dist-test-slave-754n"
server_key: "2edc19541e9f7b43b026c09ff104e0d3"
server_key_iv: "d47de327ce4b223dde70f6b671140694"
server_key_version: "encryptionkey@0"
I20260923 23:14:44.556113 31541 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 23:14:44.556798 31556 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:44.556957 31541 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:44.557051 31541 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/master-0/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/master-0/wal
uuid: "2ddae489e6844395a47c75343be04fa5"
format_stamp: "Formatted at 2026-09-23 23:14:44 on dist-test-slave-754n"
server_key: "2edc19541e9f7b43b026c09ff104e0d3"
server_key_iv: "d47de327ce4b223dde70f6b671140694"
server_key_version: "encryptionkey@0"
I20260923 23:14:44.557125 31541 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:14:44.566313 31541 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:14:44.567771 31541 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:14:44.567936 31541 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:14:44.571405 31541 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.254:41951
I20260923 23:14:44.571439 31608 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.254:41951 every 8 connection(s)
I20260923 23:14:44.571766 31541 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
I20260923 23:14:44.572759 31609 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:14:44.574839 31609 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ddae489e6844395a47c75343be04fa5: Bootstrap starting.
I20260923 23:14:44.575381 31609 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2ddae489e6844395a47c75343be04fa5: Neither blocks nor log segments found. Creating new log.
I20260923 23:14:44.575655 31609 log.cc:824] T 00000000000000000000000000000000 P 2ddae489e6844395a47c75343be04fa5: Log is configured to *not* fsync() on all Append() calls
I20260923 23:14:44.576364 31609 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ddae489e6844395a47c75343be04fa5: No bootstrap required, opened a new log
I20260923 23:14:44.577760 31609 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2ddae489e6844395a47c75343be04fa5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ddae489e6844395a47c75343be04fa5" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41951 } }
I20260923 23:14:44.577875 31609 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2ddae489e6844395a47c75343be04fa5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:14:44.577921 31609 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2ddae489e6844395a47c75343be04fa5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ddae489e6844395a47c75343be04fa5, State: Initialized, Role: FOLLOWER
I20260923 23:14:44.578069 31609 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2ddae489e6844395a47c75343be04fa5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ddae489e6844395a47c75343be04fa5" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41951 } }
I20260923 23:14:44.578140 31609 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2ddae489e6844395a47c75343be04fa5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 23:14:44.578177 31609 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2ddae489e6844395a47c75343be04fa5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 23:14:44.578220 31609 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2ddae489e6844395a47c75343be04fa5 [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:14:44.578480 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 31541
I20260923 23:14:44.578612 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/master-0/wal/instance
I20260923 23:14:44.578791 25487 external_mini_cluster.cc:1428] Setting key 04f6337e34b551699a0ceab5db2ecaf9
I20260923 23:14:44.579061 31609 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2ddae489e6844395a47c75343be04fa5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ddae489e6844395a47c75343be04fa5" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41951 } }
I20260923 23:14:44.579177 31609 leader_election.cc:302] T 00000000000000000000000000000000 P 2ddae489e6844395a47c75343be04fa5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2ddae489e6844395a47c75343be04fa5; no voters:
I20260923 23:14:44.579351 31609 leader_election.cc:288] T 00000000000000000000000000000000 P 2ddae489e6844395a47c75343be04fa5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 23:14:44.579407 31614 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2ddae489e6844395a47c75343be04fa5 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 23:14:44.579582 31614 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2ddae489e6844395a47c75343be04fa5 [term 1 LEADER]: Becoming Leader. State: Replica: 2ddae489e6844395a47c75343be04fa5, State: Running, Role: LEADER
I20260923 23:14:44.579685 31609 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2ddae489e6844395a47c75343be04fa5 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 23:14:44.579699 31614 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2ddae489e6844395a47c75343be04fa5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ddae489e6844395a47c75343be04fa5" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41951 } }
I20260923 23:14:44.581449 31615 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ddae489e6844395a47c75343be04fa5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2ddae489e6844395a47c75343be04fa5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ddae489e6844395a47c75343be04fa5" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41951 } } }
I20260923 23:14:44.581470 31616 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ddae489e6844395a47c75343be04fa5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ddae489e6844395a47c75343be04fa5. Latest consensus state: current_term: 1 leader_uuid: "2ddae489e6844395a47c75343be04fa5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ddae489e6844395a47c75343be04fa5" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41951 } } }
I20260923 23:14:44.582171 31615 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ddae489e6844395a47c75343be04fa5 [sys.catalog]: This master's current role is: LEADER
I20260923 23:14:44.582182 31616 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ddae489e6844395a47c75343be04fa5 [sys.catalog]: This master's current role is: LEADER
W20260923 23:14:44.581890 31629 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2ddae489e6844395a47c75343be04fa5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 23:14:44.582894 31629 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 23:14:44.583005 31631 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 23:14:44.583196 31631 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 23:14:44.584842 31631 catalog_manager.cc:1347] Generated new cluster ID: 25a76464fda54de4ba6067e47f542e90
I20260923 23:14:44.584897 31631 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 23:14:44 dist-test-slave-754n krb5kdc[31528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205284, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM
I20260923 23:14:44.589401 31613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:44.579228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35648 (local address 127.24.227.254:41951)
0923 23:14:44.579399 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:44.579407 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:44.579463 (+ 56us) server_negotiation.cc:407] Connection header received
0923 23:14:44.579575 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:44.579582 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:44.579636 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:44.579793 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:44.580539 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:44.581952 (+ 1413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:44.582863 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:44.583464 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:44.583586 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:44.585872 (+ 2286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:44.585876 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:44.585876 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:44.585887 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:44.587699 (+ 1812us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:44.588123 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:44.588125 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:44.588125 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:44.588236 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:44.588464 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:44.588465 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:44.588465 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:44.588735 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:44.588901 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:44.589103 (+ 202us) server_negotiation.cc:299] Negotiation successful
0923 23:14:44.589234 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":76,"threads_started":1}
I20260923 23:14:44.605854 31631 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 23:14:44.606513 31631 catalog_manager.cc:1504] Loading token signing keys...
I20260923 23:14:44.609501 31631 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2ddae489e6844395a47c75343be04fa5: Generated new TSK 0
I20260923 23:14:44.609634 31631 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.24.227.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.227.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.227.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/krb5kdc/oryx_127.24.227.193.keytab.
Entry for principal oryx/127.24.227.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/krb5kdc/oryx_127.24.227.193.keytab.
I20260923 23:14:44.638672 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.227.193:0
--local_ip_for_outbound_sockets=127.24.227.193
--webserver_interface=127.24.227.193
--webserver_port=0
--tserver_master_addrs=127.24.227.254:41951
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/krb5kdc/oryx_127.24.227.193.keytab
--principal=oryx/127.24.227.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:42907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:14:44.735767 31635 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:14:44.736065 31635 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:14:44.736104 31635 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:14:44.737203 31635 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:14:44.737267 31635 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:14:44.737308 31635 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.227.193
I20260923 23:14:44.738610 31635 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:42907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/krb5kdc/oryx_127.24.227.193.keytab
--principal=oryx/127.24.227.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.227.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.227.193
--webserver_port=0
--tserver_master_addrs=127.24.227.254:41951
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31635
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.227.193
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:14:44.738883 31635 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:14:44.739099 31635 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:14:44.743535 31640 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:44.743551 31641 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:44.743501 31643 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:14:44.743702 31635 server_base.cc:1034] running on GCE node
I20260923 23:14:44.743932 31635 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:14:44.744150 31635 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:14:44.745326 31635 hybrid_clock.cc:648] HybridClock initialized: now 1790205284745312 us; error 28 us; skew 500 ppm
Sep 23 23:14:44 dist-test-slave-754n krb5kdc[31528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205284, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.227.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:14:44.748307 31635 init.cc:377] Logged in from keytab as oryx/127.24.227.193@KRBTEST.COM (short username oryx)
I20260923 23:14:44.748682 31635 webserver.cc:460] Webserver started at http://127.24.227.193:46723/ using document root <none> and password file <none>
I20260923 23:14:44.748930 31635 fs_manager.cc:359] Metadata directory not provided
I20260923 23:14:44.748992 31635 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:14:44.749114 31635 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:14:44.750075 31635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/ts-0/data/instance:
uuid: "85902a6195e544b7a8e8794e0b7c7ca1"
format_stamp: "Formatted at 2026-09-23 23:14:44 on dist-test-slave-754n"
server_key: "c30d4b74d78050e9c47610f2306a99cc"
server_key_iv: "cb4136723a2eb13f0ea8688c9b9822a0"
server_key_version: "encryptionkey@0"
I20260923 23:14:44.750403 31635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/ts-0/wal/instance:
uuid: "85902a6195e544b7a8e8794e0b7c7ca1"
format_stamp: "Formatted at 2026-09-23 23:14:44 on dist-test-slave-754n"
server_key: "c30d4b74d78050e9c47610f2306a99cc"
server_key_iv: "cb4136723a2eb13f0ea8688c9b9822a0"
server_key_version: "encryptionkey@0"
I20260923 23:14:44.751852 31635 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260923 23:14:44.752679 31650 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:44.752833 31635 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 23:14:44.752928 31635 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/ts-0/wal
uuid: "85902a6195e544b7a8e8794e0b7c7ca1"
format_stamp: "Formatted at 2026-09-23 23:14:44 on dist-test-slave-754n"
server_key: "c30d4b74d78050e9c47610f2306a99cc"
server_key_iv: "cb4136723a2eb13f0ea8688c9b9822a0"
server_key_version: "encryptionkey@0"
I20260923 23:14:44.753002 31635 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:14:44.775359 31635 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:14:44.777639 31635 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:14:44.777796 31635 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:14:44.778024 31635 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 23:14:44.778316 31635 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 23:14:44.778385 31635 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:44.778417 31635 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 23:14:44.778443 31635 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:44.784152 31635 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.193:42775
I20260923 23:14:44.784231 31763 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.193:42775 every 8 connection(s)
I20260923 23:14:44.784531 31635 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/ts-0/data/info.pb
Sep 23 23:14:44 dist-test-slave-754n krb5kdc[31528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205284, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.227.193@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM
I20260923 23:14:44.793569 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 31635
I20260923 23:14:44.793720 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/ts-0/wal/instance
I20260923 23:14:44.793771 31613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:44.785228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.193:41943 (local address 127.24.227.254:41951)
0923 23:14:44.785273 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:44.785275 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:44.785460 (+ 185us) server_negotiation.cc:407] Connection header received
0923 23:14:44.785976 (+ 516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:44.785977 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:44.785989 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:44.786049 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:44.786947 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:44.787416 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:44.788395 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:44.788602 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:44.788684 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:44.790870 (+ 2186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:44.790871 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:44.790871 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:44.790877 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:44.792359 (+ 1482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:44.792771 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:44.792772 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:44.792772 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:44.792849 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:44.793075 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:44.793077 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:44.793077 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:44.793225 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:44.793298 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:44.793580 (+ 282us) server_negotiation.cc:299] Negotiation successful
0923 23:14:44.793684 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260923 23:14:44.793912 31766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:44.785187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41951 (local address 127.24.227.193:41943)
0923 23:14:44.785329 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:44.785374 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:44.785866 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:44.786127 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:44.786130 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:44.786680 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:14:44.786849 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:44.786854 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:44.787514 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:44.787515 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:44.788288 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:44.788290 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:44.788694 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:44.788695 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:44.788816 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:44.789255 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:14:44.789264 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:14:44.790755 (+ 1491us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:14:44.792490 (+ 1735us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:44.792496 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:44.792497 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:44.792693 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:44.792917 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:44.792917 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:44.792917 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:44.793007 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:44.793290 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:14:44.793294 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:14:44.793418 (+ 124us) client_negotiation.cc:769] Sending connection context
0923 23:14:44.793550 (+ 132us) client_negotiation.cc:240] Negotiation successful
0923 23:14:44.793700 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":63,"threads_started":1}
I20260923 23:14:44.793954 25487 external_mini_cluster.cc:1428] Setting key e927615efdaa7ac3ee5c3ad81a40b3e6
I20260923 23:14:44.794576 31764 heartbeater.cc:347] Connected to a master server at 127.24.227.254:41951
I20260923 23:14:44.794670 31764 heartbeater.cc:464] Registering TS with master...
I20260923 23:14:44.794847 31764 heartbeater.cc:511] Master 127.24.227.254:41951 requested a full tablet report, sending...
I20260923 23:14:44.795336 31572 ts_manager.cc:194] Registered new tserver with Master: 85902a6195e544b7a8e8794e0b7c7ca1 (127.24.227.193:42775)
I20260923 23:14:44.796036 31572 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.227.193@KRBTEST.COM'} at 127.24.227.193:41943
WARNING: no policy specified for oryx/127.24.227.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.227.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.227.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/krb5kdc/oryx_127.24.227.194.keytab.
Entry for principal oryx/127.24.227.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/krb5kdc/oryx_127.24.227.194.keytab.
I20260923 23:14:44.816793 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.227.194:0
--local_ip_for_outbound_sockets=127.24.227.194
--webserver_interface=127.24.227.194
--webserver_port=0
--tserver_master_addrs=127.24.227.254:41951
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/krb5kdc/oryx_127.24.227.194.keytab
--principal=oryx/127.24.227.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:42907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:14:44.893078 31769 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:14:44.893277 31769 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:14:44.893314 31769 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:14:44.894493 31769 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:14:44.894556 31769 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:14:44.894601 31769 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.227.194
I20260923 23:14:44.896041 31769 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:42907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/krb5kdc/oryx_127.24.227.194.keytab
--principal=oryx/127.24.227.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.227.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.227.194
--webserver_port=0
--tserver_master_addrs=127.24.227.254:41951
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31769
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.227.194
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:14:44.896275 31769 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:14:44.896502 31769 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:14:44.901338 31774 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:44.901332 31777 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:14:44.901376 31769 server_base.cc:1034] running on GCE node
W20260923 23:14:44.901332 31775 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:14:44.901763 31769 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:14:44.902001 31769 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:14:44.903159 31769 hybrid_clock.cc:648] HybridClock initialized: now 1790205284903158 us; error 36 us; skew 500 ppm
Sep 23 23:14:44 dist-test-slave-754n krb5kdc[31528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205284, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.227.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:14:44.905567 31769 init.cc:377] Logged in from keytab as oryx/127.24.227.194@KRBTEST.COM (short username oryx)
I20260923 23:14:44.906020 31769 webserver.cc:460] Webserver started at http://127.24.227.194:45683/ using document root <none> and password file <none>
I20260923 23:14:44.906234 31769 fs_manager.cc:359] Metadata directory not provided
I20260923 23:14:44.906306 31769 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:14:44.906454 31769 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:14:44.907418 31769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/ts-1/data/instance:
uuid: "99b029603b8e4c9fbafedaafc91ea951"
format_stamp: "Formatted at 2026-09-23 23:14:44 on dist-test-slave-754n"
server_key: "b54c31f66bc175b2320a23c8263b7533"
server_key_iv: "3c49046c2ae5e8e0f7792fed3b6447ef"
server_key_version: "encryptionkey@0"
I20260923 23:14:44.907759 31769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/ts-1/wal/instance:
uuid: "99b029603b8e4c9fbafedaafc91ea951"
format_stamp: "Formatted at 2026-09-23 23:14:44 on dist-test-slave-754n"
server_key: "b54c31f66bc175b2320a23c8263b7533"
server_key_iv: "3c49046c2ae5e8e0f7792fed3b6447ef"
server_key_version: "encryptionkey@0"
I20260923 23:14:44.909001 31769 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 23:14:44.909675 31784 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:44.909837 31769 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 23:14:44.909934 31769 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/ts-1/wal
uuid: "99b029603b8e4c9fbafedaafc91ea951"
format_stamp: "Formatted at 2026-09-23 23:14:44 on dist-test-slave-754n"
server_key: "b54c31f66bc175b2320a23c8263b7533"
server_key_iv: "3c49046c2ae5e8e0f7792fed3b6447ef"
server_key_version: "encryptionkey@0"
I20260923 23:14:44.910022 31769 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:14:44.919263 31769 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:14:44.921348 31769 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:14:44.921492 31769 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:14:44.921671 31769 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 23:14:44.921923 31769 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 23:14:44.921972 31769 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:44.922005 31769 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 23:14:44.922029 31769 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:44.927839 31769 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.194:33365
I20260923 23:14:44.927893 31897 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.194:33365 every 8 connection(s)
I20260923 23:14:44.928146 31769 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/ts-1/data/info.pb
I20260923 23:14:44.930639 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 31769
I20260923 23:14:44.930737 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/ts-1/wal/instance
I20260923 23:14:44.930934 25487 external_mini_cluster.cc:1428] Setting key 9f661bdc41eb5f98182009e20c115f19
Sep 23 23:14:44 dist-test-slave-754n krb5kdc[31528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205284, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.227.194@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM
I20260923 23:14:44.937520 31613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:44.928729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.194:53897 (local address 127.24.227.254:41951)
0923 23:14:44.928791 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:44.928793 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:44.928981 (+ 188us) server_negotiation.cc:407] Connection header received
0923 23:14:44.929548 (+ 567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:44.929549 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:44.929559 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:44.929599 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:44.930362 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:44.930969 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:44.932105 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:44.932329 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:44.932415 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:44.934589 (+ 2174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:44.934591 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:44.934591 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:44.934599 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:44.936262 (+ 1663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:44.936630 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:44.936631 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:44.936631 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:44.936692 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:44.936907 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:44.936907 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:44.936908 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:44.937067 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:44.937136 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:44.937344 (+ 208us) server_negotiation.cc:299] Negotiation successful
0923 23:14:44.937427 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260923 23:14:44.937530 31900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:44.928700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41951 (local address 127.24.227.194:53897)
0923 23:14:44.928850 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:44.928891 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:44.929322 (+ 431us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:44.929658 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:44.929661 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:44.930137 (+ 476us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:14:44.930293 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:44.930297 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:44.931043 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:44.931044 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:44.932024 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:44.932027 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:44.932429 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:44.932430 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:44.932553 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:44.933021 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:14:44.933031 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:14:44.934497 (+ 1466us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:14:44.936339 (+ 1842us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:44.936344 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:44.936345 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:44.936557 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:44.936760 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:44.936760 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:44.936761 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:44.936844 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:44.937129 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:14:44.937132 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:14:44.937225 (+ 93us) client_negotiation.cc:769] Sending connection context
0923 23:14:44.937296 (+ 71us) client_negotiation.cc:240] Negotiation successful
0923 23:14:44.937394 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":75,"threads_started":1}
I20260923 23:14:44.938079 31898 heartbeater.cc:347] Connected to a master server at 127.24.227.254:41951
I20260923 23:14:44.938174 31898 heartbeater.cc:464] Registering TS with master...
I20260923 23:14:44.938344 31898 heartbeater.cc:511] Master 127.24.227.254:41951 requested a full tablet report, sending...
I20260923 23:14:44.938808 31572 ts_manager.cc:194] Registered new tserver with Master: 99b029603b8e4c9fbafedaafc91ea951 (127.24.227.194:33365)
I20260923 23:14:44.939170 31572 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.227.194@KRBTEST.COM'} at 127.24.227.194:53897
WARNING: no policy specified for oryx/127.24.227.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.227.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.227.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/krb5kdc/oryx_127.24.227.195.keytab.
Entry for principal oryx/127.24.227.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/krb5kdc/oryx_127.24.227.195.keytab.
I20260923 23:14:44.955092 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.227.195:0
--local_ip_for_outbound_sockets=127.24.227.195
--webserver_interface=127.24.227.195
--webserver_port=0
--tserver_master_addrs=127.24.227.254:41951
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/krb5kdc/oryx_127.24.227.195.keytab
--principal=oryx/127.24.227.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:42907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:14:45.030143 31903 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:14:45.030309 31903 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:14:45.030329 31903 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:14:45.031440 31903 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:14:45.031491 31903 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:14:45.031515 31903 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.227.195
I20260923 23:14:45.032812 31903 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:42907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/krb5kdc/oryx_127.24.227.195.keytab
--principal=oryx/127.24.227.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.227.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.227.195
--webserver_port=0
--tserver_master_addrs=127.24.227.254:41951
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31903
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.227.195
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:14:45.033036 31903 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:14:45.033242 31903 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:14:45.037817 31908 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:45.037786 31909 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:14:45.037825 31903 server_base.cc:1034] running on GCE node
W20260923 23:14:45.037798 31911 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:14:45.038199 31903 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:14:45.038426 31903 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:14:45.039580 31903 hybrid_clock.cc:648] HybridClock initialized: now 1790205285039554 us; error 38 us; skew 500 ppm
Sep 23 23:14:45 dist-test-slave-754n krb5kdc[31528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205285, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.227.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:14:45.041903 31903 init.cc:377] Logged in from keytab as oryx/127.24.227.195@KRBTEST.COM (short username oryx)
I20260923 23:14:45.042320 31903 webserver.cc:460] Webserver started at http://127.24.227.195:46125/ using document root <none> and password file <none>
I20260923 23:14:45.042526 31903 fs_manager.cc:359] Metadata directory not provided
I20260923 23:14:45.042564 31903 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:14:45.042645 31903 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:14:45.043565 31903 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/ts-2/data/instance:
uuid: "6eb81312874d421c9884f64eff92eae6"
format_stamp: "Formatted at 2026-09-23 23:14:45 on dist-test-slave-754n"
server_key: "4f4e8217e5135576fe5e10d4ccbe4579"
server_key_iv: "3f24b987af94d1e044e89b73e7e49be5"
server_key_version: "encryptionkey@0"
I20260923 23:14:45.043830 31903 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/ts-2/wal/instance:
uuid: "6eb81312874d421c9884f64eff92eae6"
format_stamp: "Formatted at 2026-09-23 23:14:45 on dist-test-slave-754n"
server_key: "4f4e8217e5135576fe5e10d4ccbe4579"
server_key_iv: "3f24b987af94d1e044e89b73e7e49be5"
server_key_version: "encryptionkey@0"
I20260923 23:14:45.045058 31903 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 23:14:45.045748 31918 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:45.045912 31903 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 23:14:45.045989 31903 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/ts-2/wal
uuid: "6eb81312874d421c9884f64eff92eae6"
format_stamp: "Formatted at 2026-09-23 23:14:45 on dist-test-slave-754n"
server_key: "4f4e8217e5135576fe5e10d4ccbe4579"
server_key_iv: "3f24b987af94d1e044e89b73e7e49be5"
server_key_version: "encryptionkey@0"
I20260923 23:14:45.046037 31903 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:14:45.056830 31903 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:14:45.058225 31903 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:14:45.058311 31903 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:14:45.058470 31903 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 23:14:45.058660 31903 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 23:14:45.058686 31903 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:45.058703 31903 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 23:14:45.058714 31903 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:45.064100 31903 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.195:43865
I20260923 23:14:45.064172 32031 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.195:43865 every 8 connection(s)
I20260923 23:14:45.064477 31903 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/ts-2/data/info.pb
I20260923 23:14:45.069321 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 31903
I20260923 23:14:45.069401 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/ts-2/wal/instance
I20260923 23:14:45.069566 25487 external_mini_cluster.cc:1428] Setting key 6564a83dcf397f5cd4743afee6946f53
Sep 23 23:14:45 dist-test-slave-754n krb5kdc[31528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205285, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.227.195@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM
I20260923 23:14:45.072433 31613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:45.065141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.195:40207 (local address 127.24.227.254:41951)
0923 23:14:45.065203 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:45.065205 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:45.065371 (+ 166us) server_negotiation.cc:407] Connection header received
0923 23:14:45.065768 (+ 397us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:45.065769 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:45.065777 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:45.065825 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:45.066542 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:45.066999 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:45.067938 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:45.068193 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:45.068243 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:45.070046 (+ 1803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:45.070047 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:45.070047 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:45.070053 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:45.071420 (+ 1367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:45.071720 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:45.071721 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:45.071722 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:45.071766 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:45.071987 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:45.071987 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:45.071988 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:45.072069 (+ 81us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:45.072112 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:45.072295 (+ 183us) server_negotiation.cc:299] Negotiation successful
0923 23:14:45.072351 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260923 23:14:45.072543 32034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:45.065121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41951 (local address 127.24.227.195:40207)
0923 23:14:45.065255 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:45.065290 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:45.065701 (+ 411us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:45.065865 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:45.065873 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:45.066326 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:14:45.066478 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:45.066481 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:45.067073 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:45.067073 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:45.067821 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:45.067822 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:45.068257 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:45.068257 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:45.068347 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:45.068705 (+ 358us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:14:45.068712 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:14:45.069987 (+ 1275us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:14:45.071481 (+ 1494us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:45.071485 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:45.071487 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:45.071665 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:45.071814 (+ 149us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:45.071814 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:45.071814 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:45.071935 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:45.072114 (+ 179us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:14:45.072118 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:14:45.072202 (+ 84us) client_negotiation.cc:769] Sending connection context
0923 23:14:45.072268 (+ 66us) client_negotiation.cc:240] Negotiation successful
0923 23:14:45.072356 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":73,"threads_started":1}
I20260923 23:14:45.072925 32032 heartbeater.cc:347] Connected to a master server at 127.24.227.254:41951
I20260923 23:14:45.073017 32032 heartbeater.cc:464] Registering TS with master...
I20260923 23:14:45.073186 32032 heartbeater.cc:511] Master 127.24.227.254:41951 requested a full tablet report, sending...
I20260923 23:14:45.073504 31572 ts_manager.cc:194] Registered new tserver with Master: 6eb81312874d421c9884f64eff92eae6 (127.24.227.195:43865)
I20260923 23:14:45.073827 31572 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.227.195@KRBTEST.COM'} at 127.24.227.195:40207
I20260923 23:14:45.081188 25487 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.24.227.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.227.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.227.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/krb5kdc/oryx_127.24.227.253.keytab.
Entry for principal oryx/127.24.227.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/krb5kdc/oryx_127.24.227.253.keytab.
I20260923 23:14:45.104120 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.253:35161
--webserver_interface=127.24.227.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/krb5kdc/oryx_127.24.227.253.keytab
--principal=oryx/127.24.227.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:42907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.227.254:41951,127.24.227.253:35161
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:14:45.179104 32037 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:14:45.179311 32037 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:14:45.179345 32037 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:14:45.180554 32037 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 23:14:45.180608 32037 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:14:45.180629 32037 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:14:45.180653 32037 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 23:14:45.180696 32037 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 23:14:45.182128 32037 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:42907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.24.227.254:41951,127.24.227.253:35161
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/krb5kdc/oryx_127.24.227.253.keytab
--principal=oryx/127.24.227.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.253:35161
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.24.227.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32037
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
W20260923 23:14:45.182248 32037 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.24.227.254:41951,127.24.227.253:35161'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260923 23:14:45.182426 32037 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:14:45.182646 32037 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:14:45.187088 32045 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:45.187095 32042 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:45.187110 32043 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:14:45.187109 32037 server_base.cc:1034] running on GCE node
I20260923 23:14:45.187454 32037 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:14:45.187668 32037 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:14:45.188817 32037 hybrid_clock.cc:648] HybridClock initialized: now 1790205285188792 us; error 37 us; skew 500 ppm
Sep 23 23:14:45 dist-test-slave-754n krb5kdc[31528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205285, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.227.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:14:45.190938 32037 init.cc:377] Logged in from keytab as oryx/127.24.227.253@KRBTEST.COM (short username oryx)
I20260923 23:14:45.191298 32037 webserver.cc:460] Webserver started at http://127.24.227.253:42639/ using document root <none> and password file <none>
I20260923 23:14:45.191507 32037 fs_manager.cc:359] Metadata directory not provided
I20260923 23:14:45.191569 32037 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:14:45.191704 32037 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:14:45.192616 32037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/master-1/data/instance:
uuid: "bd675df8e6274c688bb00c0f39e018c1"
format_stamp: "Formatted at 2026-09-23 23:14:45 on dist-test-slave-754n"
server_key: "6dad79b0568eaac441ff9e000823a40f"
server_key_iv: "ed60b6f15e6eed7acc97be5bc36f81a3"
server_key_version: "encryptionkey@0"
I20260923 23:14:45.192976 32037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/master-1/wal/instance:
uuid: "bd675df8e6274c688bb00c0f39e018c1"
format_stamp: "Formatted at 2026-09-23 23:14:45 on dist-test-slave-754n"
server_key: "6dad79b0568eaac441ff9e000823a40f"
server_key_iv: "ed60b6f15e6eed7acc97be5bc36f81a3"
server_key_version: "encryptionkey@0"
I20260923 23:14:45.194197 32037 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 23:14:45.194877 32052 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:45.195037 32037 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 23:14:45.195132 32037 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/master-1/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/master-1/wal
uuid: "bd675df8e6274c688bb00c0f39e018c1"
format_stamp: "Formatted at 2026-09-23 23:14:45 on dist-test-slave-754n"
server_key: "6dad79b0568eaac441ff9e000823a40f"
server_key_iv: "ed60b6f15e6eed7acc97be5bc36f81a3"
server_key_version: "encryptionkey@0"
I20260923 23:14:45.195216 32037 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:14:45.212772 32037 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:14:45.214865 32037 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:14:45.215022 32037 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:14:45.218616 32037 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.253:35161
I20260923 23:14:45.218642 32104 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.253:35161 every 8 connection(s)
I20260923 23:14:45.218921 32037 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/master-1/data/info.pb
I20260923 23:14:45.219810 32105 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:14:45.221160 32105 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41951 } has no permanent_uuid. Determining permanent_uuid...
Sep 23 23:14:45 dist-test-slave-754n krb5kdc[31528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205285, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.227.253@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM
I20260923 23:14:45.228225 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 32037
I20260923 23:14:45.229053 31613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:45.221763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35662 (local address 127.24.227.254:41951)
0923 23:14:45.221820 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:45.221822 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:45.221973 (+ 151us) server_negotiation.cc:407] Connection header received
0923 23:14:45.222336 (+ 363us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:45.222336 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:45.222345 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:45.222402 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:45.222935 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:45.223402 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:45.224339 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:45.224520 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:45.224572 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:45.226350 (+ 1778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:45.226351 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:45.226351 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:45.226356 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:45.227603 (+ 1247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:45.227976 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:45.227977 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:45.227978 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:45.228067 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:45.228299 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:45.228301 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:45.228301 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:45.228515 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:45.228582 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:45.228882 (+ 300us) server_negotiation.cc:299] Negotiation successful
0923 23:14:45.228956 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260923 23:14:45.229207 32107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:45.221729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41951 (local address 127.0.0.1:35662)
0923 23:14:45.221865 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:45.221898 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:45.222268 (+ 370us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:45.222484 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:45.222487 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:45.222698 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:14:45.222866 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:45.222869 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:45.223474 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:45.223475 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:45.224265 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:45.224267 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:45.224585 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:45.224586 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:45.224681 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:45.225061 (+ 380us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:14:45.225069 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:14:45.226281 (+ 1212us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:14:45.227663 (+ 1382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:45.227667 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:45.227669 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:45.227861 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:45.228136 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:45.228137 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:45.228137 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:45.228231 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:45.228592 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:14:45.228598 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:14:45.228753 (+ 155us) client_negotiation.cc:769] Sending connection context
0923 23:14:45.228858 (+ 105us) client_negotiation.cc:240] Negotiation successful
0923 23:14:45.229003 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":75,"threads_started":1}
I20260923 23:14:45.229784 32105 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.227.253" port: 35161 } has no permanent_uuid. Determining permanent_uuid...
Sep 23 23:14:45 dist-test-slave-754n krb5kdc[31528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205285, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.227.253@KRBTEST.COM for oryx/127.24.227.253@KRBTEST.COM
Sep 23 23:14:45 dist-test-slave-754n krb5kdc[31528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205284, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.227.253@KRBTEST.COM
I20260923 23:14:45.236334 32107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:45.229996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.253:35161 (local address 127.0.0.1:50588)
0923 23:14:45.230035 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:45.230044 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:45.230074 (+ 30us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:45.230262 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:45.230263 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:45.230383 (+ 120us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:14:45.230416 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:45.230417 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:45.230966 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:45.230967 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:45.231413 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:45.231414 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:45.231756 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:45.231757 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:45.231807 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:45.232200 (+ 393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:14:45.232204 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:14:45.233293 (+ 1089us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:14:45.235289 (+ 1996us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:45.235290 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:45.235290 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:45.235455 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:45.235680 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:45.235681 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:45.235681 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:45.235735 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:45.236036 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:14:45.236037 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:14:45.236072 (+ 35us) client_negotiation.cc:769] Sending connection context
0923 23:14:45.236143 (+ 71us) client_negotiation.cc:240] Negotiation successful
0923 23:14:45.236230 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26,"mutex_wait_us":18}
I20260923 23:14:45.236394 32110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:45.230045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50588 (local address 127.24.227.253:35161)
0923 23:14:45.230168 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:45.230170 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:45.230180 (+ 10us) server_negotiation.cc:407] Connection header received
0923 23:14:45.230192 (+ 12us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:45.230192 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:45.230198 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:45.230261 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:45.230492 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:45.230900 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:45.231489 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:45.231685 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:45.231756 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:45.233391 (+ 1635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:45.233397 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:45.233397 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:45.233403 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:45.235183 (+ 1780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:45.235549 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:45.235552 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:45.235552 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:45.235611 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:45.235797 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:45.235798 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:45.235798 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:45.235972 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:45.236075 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:45.236172 (+ 97us) server_negotiation.cc:299] Negotiation successful
0923 23:14:45.236278 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":44,"server-negotiator.queue_time_us":107,"thread_start_us":72,"threads_started":1}
I20260923 23:14:45.237545 32109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:45.228699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50574 (local address 127.24.227.253:35161)
0923 23:14:45.228841 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:45.228845 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:45.228910 (+ 65us) server_negotiation.cc:407] Connection header received
0923 23:14:45.228980 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:45.228983 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:45.228995 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:45.229089 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:45.229740 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:45.230521 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:45.231191 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:45.231483 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:45.231566 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:45.233959 (+ 2393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:45.233960 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:45.233961 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:45.233968 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:45.236183 (+ 2215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:45.236770 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:45.236771 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:45.236771 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:45.236829 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:45.237044 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:45.237045 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:45.237046 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:45.237154 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:45.237278 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:45.237343 (+ 65us) server_negotiation.cc:299] Negotiation successful
0923 23:14:45.237440 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":118,"thread_start_us":71,"threads_started":1}
I20260923 23:14:45.237800 32105 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd675df8e6274c688bb00c0f39e018c1: Bootstrap starting.
I20260923 23:14:45.238353 32105 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bd675df8e6274c688bb00c0f39e018c1: Neither blocks nor log segments found. Creating new log.
I20260923 23:14:45.238602 32105 log.cc:824] T 00000000000000000000000000000000 P bd675df8e6274c688bb00c0f39e018c1: Log is configured to *not* fsync() on all Append() calls
I20260923 23:14:45.239166 32105 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd675df8e6274c688bb00c0f39e018c1: No bootstrap required, opened a new log
I20260923 23:14:45.240481 32105 raft_consensus.cc:348] T 00000000000000000000000000000000 P bd675df8e6274c688bb00c0f39e018c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ddae489e6844395a47c75343be04fa5" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41951 } } peers { permanent_uuid: "bd675df8e6274c688bb00c0f39e018c1" member_type: VOTER last_known_addr { host: "127.24.227.253" port: 35161 } }
I20260923 23:14:45.240584 32105 raft_consensus.cc:374] T 00000000000000000000000000000000 P bd675df8e6274c688bb00c0f39e018c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:14:45.240605 32105 raft_consensus.cc:729] T 00000000000000000000000000000000 P bd675df8e6274c688bb00c0f39e018c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd675df8e6274c688bb00c0f39e018c1, State: Initialized, Role: FOLLOWER
I20260923 23:14:45.240711 32105 consensus_queue.cc:260] T 00000000000000000000000000000000 P bd675df8e6274c688bb00c0f39e018c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ddae489e6844395a47c75343be04fa5" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41951 } } peers { permanent_uuid: "bd675df8e6274c688bb00c0f39e018c1" member_type: VOTER last_known_addr { host: "127.24.227.253" port: 35161 } }
I20260923 23:14:45.240932 32112 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd675df8e6274c688bb00c0f39e018c1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ddae489e6844395a47c75343be04fa5" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41951 } } peers { permanent_uuid: "bd675df8e6274c688bb00c0f39e018c1" member_type: VOTER last_known_addr { host: "127.24.227.253" port: 35161 } } }
I20260923 23:14:45.240984 32105 sys_catalog.cc:564] T 00000000000000000000000000000000 P bd675df8e6274c688bb00c0f39e018c1 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 23:14:45.241027 32112 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd675df8e6274c688bb00c0f39e018c1 [sys.catalog]: This master's current role is: FOLLOWER
W20260923 23:14:45.242566 32123 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bd675df8e6274c688bb00c0f39e018c1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 23:14:45.242627 32123 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 23:14:45.243825 32037 master_runner.cc:428] Detected that this master bd675df8e6274c688bb00c0f39e018c1 is joining an existing cluster
I20260923 23:14:45.243893 32037 master_runner.cc:432] Initiating AddMaster RPC to add 127.24.227.253:35161
I20260923 23:14:45.250914 32130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:45.244864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41951 (local address 127.0.0.1:35668)
0923 23:14:45.245058 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:45.245069 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:45.245130 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:45.245279 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:45.245280 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:45.245518 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:14:45.245597 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:45.245599 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:45.246241 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:45.246242 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:45.247084 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:45.247086 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:45.247509 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:45.247509 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:45.247548 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:45.247964 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:14:45.247968 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:14:45.248456 (+ 488us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:14:45.249874 (+ 1418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:45.249875 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:45.249876 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:45.250057 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:45.250255 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:45.250256 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:45.250256 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:45.250320 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:45.250581 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:14:45.250582 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:14:45.250621 (+ 39us) client_negotiation.cc:769] Sending connection context
0923 23:14:45.250723 (+ 102us) client_negotiation.cc:240] Negotiation successful
0923 23:14:45.250816 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":82,"threads_started":1}
I20260923 23:14:45.250900 31613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:45.244973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35668 (local address 127.24.227.254:41951)
0923 23:14:45.245013 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:45.245015 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:45.245137 (+ 122us) server_negotiation.cc:407] Connection header received
0923 23:14:45.245187 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:45.245188 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:45.245197 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:45.245224 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:45.245693 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:45.246135 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:45.247178 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:45.247415 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:45.247446 (+ 31us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:45.248553 (+ 1107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:45.248555 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:45.248555 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:45.248561 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:45.249798 (+ 1237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:45.250136 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:45.250137 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:45.250137 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:45.250194 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:45.250385 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:45.250386 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:45.250386 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:45.250510 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:45.250577 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:45.250748 (+ 171us) server_negotiation.cc:299] Negotiation successful
0923 23:14:45.250816 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260923 23:14:45.251920 32129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:45.244861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.253:35161 (local address 127.0.0.1:50590)
0923 23:14:45.245058 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:45.245069 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:45.245130 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:45.245269 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:45.245270 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:45.245546 (+ 276us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:14:45.245597 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:45.245599 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:45.246457 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:45.246458 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:45.247214 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:45.247215 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:45.247670 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:45.247671 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:45.247726 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:45.248118 (+ 392us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:14:45.248123 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:14:45.248514 (+ 391us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:14:45.250787 (+ 2273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:45.250788 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:45.250788 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:45.250973 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:45.251205 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:45.251206 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:45.251206 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:45.251285 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:45.251557 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:14:45.251558 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:14:45.251590 (+ 32us) client_negotiation.cc:769] Sending connection context
0923 23:14:45.251651 (+ 61us) client_negotiation.cc:240] Negotiation successful
0923 23:14:45.251767 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"spinlock_wait_cycles":9088,"thread_start_us":108,"threads_started":1}
I20260923 23:14:45.251987 32109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:45.244936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50590 (local address 127.24.227.253:35161)
0923 23:14:45.245016 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:45.245018 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:45.245165 (+ 147us) server_negotiation.cc:407] Connection header received
0923 23:14:45.245184 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:45.245185 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:45.245194 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:45.245262 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:45.245681 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:45.246373 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:45.247286 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:45.247585 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:45.247654 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:45.248603 (+ 949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:45.248604 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:45.248605 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:45.248610 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:45.250705 (+ 2095us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:45.251061 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:45.251062 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:45.251063 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:45.251132 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:45.251360 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:45.251361 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:45.251361 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:45.251480 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:45.251519 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:45.251685 (+ 166us) server_negotiation.cc:299] Negotiation successful
0923 23:14:45.251832 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":66,"server-negotiator.queue_time_us":64}
Sep 23 23:14:45 dist-test-slave-754n krb5kdc[31528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205284, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.227.254@KRBTEST.COM for oryx/127.24.227.253@KRBTEST.COM
I20260923 23:14:45.259967 32109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:45.253670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50602 (local address 127.24.227.253:35161)
0923 23:14:45.253707 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:45.253709 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:45.253823 (+ 114us) server_negotiation.cc:407] Connection header received
0923 23:14:45.253917 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:45.253918 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:45.253927 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:45.253994 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:45.254393 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:45.254828 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:45.255649 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:45.255832 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:45.255927 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:45.257617 (+ 1690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:45.257618 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:45.257618 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:45.257624 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:45.258796 (+ 1172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:45.259172 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:45.259173 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:45.259174 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:45.259229 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:45.259482 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:45.259482 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:45.259483 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:45.259569 (+ 86us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:45.259619 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:45.259800 (+ 181us) server_negotiation.cc:299] Negotiation successful
0923 23:14:45.259852 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 23:14:45.260018 32131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:45.253598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.253:35161 (local address 127.0.0.1:50602)
0923 23:14:45.253723 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:45.253756 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:45.253846 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:45.253999 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:45.254002 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:45.254246 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:14:45.254311 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:45.254313 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:45.254909 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:45.254910 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:45.255571 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:45.255573 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:45.255953 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:45.255955 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:45.256018 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:45.256465 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:14:45.256469 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:14:45.257539 (+ 1070us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:14:45.258878 (+ 1339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:45.258884 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:45.258886 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:45.259080 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:45.259330 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:45.259332 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:45.259332 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:45.259403 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:45.259634 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:14:45.259640 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:14:45.259689 (+ 49us) client_negotiation.cc:769] Sending connection context
0923 23:14:45.259776 (+ 87us) client_negotiation.cc:240] Negotiation successful
0923 23:14:45.259923 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":76,"threads_started":1}
I20260923 23:14:45.260797 31572 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.24.227.253:35161: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "bd675df8e6274c688bb00c0f39e018c1"
member_type: NON_VOTER
last_known_addr {
host: "127.24.227.253"
port: 35161
}
attrs {
promote: true
}
}
dest_uuid: "2ddae489e6844395a47c75343be04fa5"
cas_config_opid_index: -1
I20260923 23:14:45.261068 31572 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2ddae489e6844395a47c75343be04fa5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "2ddae489e6844395a47c75343be04fa5" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41951 } } peers { permanent_uuid: "bd675df8e6274c688bb00c0f39e018c1" member_type: NON_VOTER last_known_addr { host: "127.24.227.253" port: 35161 } attrs { promote: true } }
I20260923 23:14:45.261459 32133 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 2ddae489e6844395a47c75343be04fa5 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER bd675df8e6274c688bb00c0f39e018c1 (127.24.227.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "2ddae489e6844395a47c75343be04fa5" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41951 } } peers { permanent_uuid: "bd675df8e6274c688bb00c0f39e018c1" member_type: NON_VOTER last_known_addr { host: "127.24.227.253" port: 35161 } attrs { promote: true } } }
I20260923 23:14:45.261734 32079 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bd675df8e6274c688bb00c0f39e018c1 [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:14:45.262403 32079 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bd675df8e6274c688bb00c0f39e018c1 [term 1 FOLLOWER]: Refusing update from remote peer 2ddae489e6844395a47c75343be04fa5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260923 23:14:45.262744 32132 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2ddae489e6844395a47c75343be04fa5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd675df8e6274c688bb00c0f39e018c1" member_type: NON_VOTER last_known_addr { host: "127.24.227.253" port: 35161 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 23:14:45.264564 32079 raft_consensus.cc:2932] T 00000000000000000000000000000000 P bd675df8e6274c688bb00c0f39e018c1 [term 1 LEARNER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER bd675df8e6274c688bb00c0f39e018c1 (127.24.227.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "2ddae489e6844395a47c75343be04fa5" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41951 } } peers { permanent_uuid: "bd675df8e6274c688bb00c0f39e018c1" member_type: NON_VOTER last_known_addr { host: "127.24.227.253" port: 35161 } attrs { promote: true } } }
I20260923 23:14:45.265690 32140 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd675df8e6274c688bb00c0f39e018c1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2ddae489e6844395a47c75343be04fa5" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "2ddae489e6844395a47c75343be04fa5" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41951 } } peers { permanent_uuid: "bd675df8e6274c688bb00c0f39e018c1" member_type: NON_VOTER last_known_addr { host: "127.24.227.253" port: 35161 } attrs { promote: true } } }
I20260923 23:14:45.265800 32140 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd675df8e6274c688bb00c0f39e018c1 [sys.catalog]: This master's current role is: LEARNER
I20260923 23:14:45.265889 32133 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.24.227.253:35161
I20260923 23:14:45.265903 32112 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd675df8e6274c688bb00c0f39e018c1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ddae489e6844395a47c75343be04fa5. Latest consensus state: current_term: 1 leader_uuid: "2ddae489e6844395a47c75343be04fa5" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "2ddae489e6844395a47c75343be04fa5" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41951 } } peers { permanent_uuid: "bd675df8e6274c688bb00c0f39e018c1" member_type: NON_VOTER last_known_addr { host: "127.24.227.253" port: 35161 } attrs { promote: true } } }
I20260923 23:14:45.265987 32112 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd675df8e6274c688bb00c0f39e018c1 [sys.catalog]: This master's current role is: LEARNER
I20260923 23:14:45.266000 32133 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ddae489e6844395a47c75343be04fa5 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "2ddae489e6844395a47c75343be04fa5" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "2ddae489e6844395a47c75343be04fa5" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41951 } } peers { permanent_uuid: "bd675df8e6274c688bb00c0f39e018c1" member_type: NON_VOTER last_known_addr { host: "127.24.227.253" port: 35161 } attrs { promote: true } } }
I20260923 23:14:45.266064 32133 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ddae489e6844395a47c75343be04fa5 [sys.catalog]: This master's current role is: LEADER
I20260923 23:14:45.266243 32037 master.cc:537] Master@127.24.227.253:35161 shutting down...
I20260923 23:14:45.266294 32136 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd675df8e6274c688bb00c0f39e018c1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2ddae489e6844395a47c75343be04fa5" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "2ddae489e6844395a47c75343be04fa5" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41951 } } peers { permanent_uuid: "bd675df8e6274c688bb00c0f39e018c1" member_type: NON_VOTER last_known_addr { host: "127.24.227.253" port: 35161 } attrs { promote: true } } }
I20260923 23:14:45.266388 32136 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd675df8e6274c688bb00c0f39e018c1 [sys.catalog]: This master's current role is: LEARNER
I20260923 23:14:45.268952 32037 raft_consensus.cc:2229] T 00000000000000000000000000000000 P bd675df8e6274c688bb00c0f39e018c1 [term 1 LEARNER]: Raft consensus shutting down.
I20260923 23:14:45.269061 32037 raft_consensus.cc:2258] T 00000000000000000000000000000000 P bd675df8e6274c688bb00c0f39e018c1 [term 1 LEARNER]: Raft consensus is shut down!
I20260923 23:14:45.269098 32037 tablet_replica.cc:330] T 00000000000000000000000000000000 P bd675df8e6274c688bb00c0f39e018c1: stopping tablet replica
I20260923 23:14:45.281641 32037 master.cc:552] Master@127.24.227.253:35161 shutdown complete.
I20260923 23:14:45.281680 32037 master_runner.cc:305] Clearing existing system tablet
I20260923 23:14:45.281821 32037 fs_manager.cc:359] Metadata directory not provided
I20260923 23:14:45.281864 32037 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:14:45.282404 32037 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260923 23:14:45.282917 32143 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:45.283028 32037 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:14:45.283084 32037 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:45.283106 32037 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/master-1/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/master-1/wal
uuid: "bd675df8e6274c688bb00c0f39e018c1"
format_stamp: "Formatted at 2026-09-23 23:14:45 on dist-test-slave-754n"
server_key: "6dad79b0568eaac441ff9e000823a40f"
server_key_iv: "ed60b6f15e6eed7acc97be5bc36f81a3"
server_key_version: "encryptionkey@0"
I20260923 23:14:45.283630 32037 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P bd675df8e6274c688bb00c0f39e018c1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 23:14:45.284971 32037 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P bd675df8e6274c688bb00c0f39e018c1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260923 23:14:45.285025 32037 log.cc:1196] T 00000000000000000000000000000000 P bd675df8e6274c688bb00c0f39e018c1: Deleting WAL directory at /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260923 23:14:45.285266 32037 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P bd675df8e6274c688bb00c0f39e018c1: Deleting consensus metadata
I20260923 23:14:45.285382 32037 master_runner.cc:315] Copying system tablet from 127.24.227.254:41951
I20260923 23:14:45.285979 32037 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P bd675df8e6274c688bb00c0f39e018c1: tablet copy: Beginning tablet copy session from remote peer at address 127.24.227.254:41951
I20260923 23:14:45.291117 31613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:45.286308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35670 (local address 127.24.227.254:41951)
0923 23:14:45.286351 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:45.286353 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:45.286401 (+ 48us) server_negotiation.cc:407] Connection header received
0923 23:14:45.286462 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:45.286463 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:45.286471 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:45.286521 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:45.286791 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:45.287221 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:45.287902 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:45.288112 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:45.288183 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:45.289034 (+ 851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:45.289035 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:45.289035 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:45.289042 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:45.290151 (+ 1109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:45.290465 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:45.290466 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:45.290467 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:45.290513 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:45.290698 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:45.290699 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:45.290699 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:45.290792 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:45.290863 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:45.290987 (+ 124us) server_negotiation.cc:299] Negotiation successful
0923 23:14:45.291042 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260923 23:14:45.291129 32149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:45.286246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41951 (local address 127.0.0.1:35670)
0923 23:14:45.286347 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:45.286358 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:45.286415 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:45.286526 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:45.286527 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:45.286671 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:14:45.286713 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:45.286714 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:45.287303 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:45.287304 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:45.287789 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:45.287790 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:45.288193 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:45.288194 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:45.288253 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:45.288701 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:14:45.288705 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:14:45.288950 (+ 245us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:14:45.290229 (+ 1279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:45.290230 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:45.290230 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:45.290398 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:45.290577 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:45.290578 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:45.290578 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:45.290627 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:45.290856 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:14:45.290857 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:14:45.290893 (+ 36us) client_negotiation.cc:769] Sending connection context
0923 23:14:45.290954 (+ 61us) client_negotiation.cc:240] Negotiation successful
0923 23:14:45.291041 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":79,"thread_start_us":61,"threads_started":1}
I20260923 23:14:45.291535 31593 tablet_copy_service.cc:140] P 2ddae489e6844395a47c75343be04fa5: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer bd675df8e6274c688bb00c0f39e018c1 ({username='oryx', principal='oryx/127.24.227.253@KRBTEST.COM'} at 127.0.0.1:35670)
I20260923 23:14:45.291621 31593 tablet_copy_service.cc:161] P 2ddae489e6844395a47c75343be04fa5: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer bd675df8e6274c688bb00c0f39e018c1 at {username='oryx', principal='oryx/127.24.227.253@KRBTEST.COM'} at 127.0.0.1:35670: session id = bd675df8e6274c688bb00c0f39e018c1-00000000000000000000000000000000
I20260923 23:14:45.292374 31593 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 2ddae489e6844395a47c75343be04fa5: Tablet Copy: opened 0 blocks and 1 log segments
I20260923 23:14:45.292919 32037 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260923 23:14:45.293995 32037 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P bd675df8e6274c688bb00c0f39e018c1: tablet copy: Starting download of 0 data blocks...
I20260923 23:14:45.294130 32037 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P bd675df8e6274c688bb00c0f39e018c1: tablet copy: Starting download of 1 WAL segments...
I20260923 23:14:45.294910 32037 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P bd675df8e6274c688bb00c0f39e018c1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260923 23:14:45.295665 31593 tablet_copy_service.cc:342] P 2ddae489e6844395a47c75343be04fa5: Request end of tablet copy session bd675df8e6274c688bb00c0f39e018c1-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.24.227.253@KRBTEST.COM'} at 127.0.0.1:35670
I20260923 23:14:45.295727 31593 tablet_copy_service.cc:433] P 2ddae489e6844395a47c75343be04fa5: ending tablet copy session bd675df8e6274c688bb00c0f39e018c1-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer bd675df8e6274c688bb00c0f39e018c1
I20260923 23:14:45.297716 32037 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:14:45.298916 32151 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:45.298870 32153 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:45.298866 32150 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:14:45.300688 32037 server_base.cc:1034] running on GCE node
I20260923 23:14:45.300787 32037 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:14:45.300905 32037 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:14:45.302037 32037 hybrid_clock.cc:648] HybridClock initialized: now 1790205285302027 us; error 41 us; skew 500 ppm
Sep 23 23:14:45 dist-test-slave-754n krb5kdc[31528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205285, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.227.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:14:45.303208 32037 init.cc:377] Logged in from keytab as oryx/127.24.227.253@KRBTEST.COM (short username oryx)
I20260923 23:14:45.303486 32037 webserver.cc:460] Webserver started at http://127.24.227.253:39653/ using document root <none> and password file <none>
I20260923 23:14:45.303599 32037 fs_manager.cc:359] Metadata directory not provided
I20260923 23:14:45.303664 32037 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:14:45.304311 32037 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260923 23:14:45.304819 32160 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:45.304957 32037 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 23:14:45.305004 32037 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/master-1/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/master-1/wal
uuid: "bd675df8e6274c688bb00c0f39e018c1"
format_stamp: "Formatted at 2026-09-23 23:14:45 on dist-test-slave-754n"
server_key: "6dad79b0568eaac441ff9e000823a40f"
server_key_iv: "ed60b6f15e6eed7acc97be5bc36f81a3"
server_key_version: "encryptionkey@0"
I20260923 23:14:45.305053 32037 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:14:45.308537 32037 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:14:45.309912 32037 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:14:45.312784 32037 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.253:35161
I20260923 23:14:45.313148 32212 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.253:35161 every 8 connection(s)
I20260923 23:14:45.313148 32037 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/master-1/data/info.pb
I20260923 23:14:45.313921 32213 sys_catalog.cc:263] Verifying existing consensus state
I20260923 23:14:45.314360 32213 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd675df8e6274c688bb00c0f39e018c1: Bootstrap starting.
I20260923 23:14:45.316423 32213 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd675df8e6274c688bb00c0f39e018c1: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 23:14:45.316625 32213 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd675df8e6274c688bb00c0f39e018c1: Bootstrap complete.
I20260923 23:14:45.316779 32213 raft_consensus.cc:348] T 00000000000000000000000000000000 P bd675df8e6274c688bb00c0f39e018c1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "2ddae489e6844395a47c75343be04fa5" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41951 } } peers { permanent_uuid: "bd675df8e6274c688bb00c0f39e018c1" member_type: NON_VOTER last_known_addr { host: "127.24.227.253" port: 35161 } attrs { promote: true } }
I20260923 23:14:45.316848 32213 raft_consensus.cc:729] T 00000000000000000000000000000000 P bd675df8e6274c688bb00c0f39e018c1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: bd675df8e6274c688bb00c0f39e018c1, State: Initialized, Role: LEARNER
I20260923 23:14:45.316879 32213 consensus_queue.cc:260] T 00000000000000000000000000000000 P bd675df8e6274c688bb00c0f39e018c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "2ddae489e6844395a47c75343be04fa5" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41951 } } peers { permanent_uuid: "bd675df8e6274c688bb00c0f39e018c1" member_type: NON_VOTER last_known_addr { host: "127.24.227.253" port: 35161 } attrs { promote: true } }
I20260923 23:14:45.317042 32213 sys_catalog.cc:564] T 00000000000000000000000000000000 P bd675df8e6274c688bb00c0f39e018c1 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 23:14:45.317027 32216 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd675df8e6274c688bb00c0f39e018c1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "2ddae489e6844395a47c75343be04fa5" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41951 } } peers { permanent_uuid: "bd675df8e6274c688bb00c0f39e018c1" member_type: NON_VOTER last_known_addr { host: "127.24.227.253" port: 35161 } attrs { promote: true } } }
I20260923 23:14:45.317145 32216 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd675df8e6274c688bb00c0f39e018c1 [sys.catalog]: This master's current role is: LEARNER
I20260923 23:14:45.318430 32227 catalog_manager.cc:1259] Loaded cluster ID: 25a76464fda54de4ba6067e47f542e90
I20260923 23:14:45.318490 32227 catalog_manager.cc:1552] T 00000000000000000000000000000000 P bd675df8e6274c688bb00c0f39e018c1: loading cluster ID for follower catalog manager: success
I20260923 23:14:45.319370 32227 catalog_manager.cc:1574] T 00000000000000000000000000000000 P bd675df8e6274c688bb00c0f39e018c1: acquiring CA information for follower catalog manager: success
I20260923 23:14:45.319619 32227 catalog_manager.cc:1602] T 00000000000000000000000000000000 P bd675df8e6274c688bb00c0f39e018c1: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260923 23:14:45.345460 32228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:45.338807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50626 (local address 127.24.227.253:35161)
0923 23:14:45.338910 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:45.338912 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:45.338917 (+ 5us) server_negotiation.cc:407] Connection header received
0923 23:14:45.338932 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:45.338932 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:45.338939 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:45.338989 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:45.339672 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:45.340323 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:45.341428 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:45.341609 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:45.341652 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:45.342984 (+ 1332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:45.342985 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:45.342985 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:45.342991 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:45.344365 (+ 1374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:45.344744 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:45.344745 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:45.344746 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:45.344793 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:45.344997 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:45.344998 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:45.344998 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:45.345085 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:45.345152 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:45.345316 (+ 164us) server_negotiation.cc:299] Negotiation successful
0923 23:14:45.345370 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82,"thread_start_us":48,"threads_started":1}
WARNING: no policy specified for oryx/127.24.227.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.227.252@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.227.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/krb5kdc/oryx_127.24.227.252.keytab.
Entry for principal oryx/127.24.227.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/krb5kdc/oryx_127.24.227.252.keytab.
I20260923 23:14:45.366348 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.252:46859
--webserver_interface=127.24.227.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/krb5kdc/oryx_127.24.227.252.keytab
--principal=oryx/127.24.227.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:42907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.227.254:41951,127.24.227.253:35161,127.24.227.252:46859
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:14:45.443005 32231 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:14:45.443178 32231 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:14:45.443200 32231 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:14:45.444300 32231 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 23:14:45.444356 32231 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:14:45.444370 32231 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:14:45.444383 32231 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 23:14:45.444396 32231 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 23:14:45.445761 32231 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:42907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.24.227.254:41951,127.24.227.253:35161,127.24.227.252:46859
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/krb5kdc/oryx_127.24.227.252.keytab
--principal=oryx/127.24.227.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.252:46859
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.24.227.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32231
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:14:45.445915 32231 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:14:45.446058 32231 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:14:45.450263 32236 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:45.450265 32237 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:45.450275 32239 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:14:45.450515 32231 server_base.cc:1034] running on GCE node
I20260923 23:14:45.450745 32231 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:14:45.451042 32231 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:14:45.452191 32231 hybrid_clock.cc:648] HybridClock initialized: now 1790205285452181 us; error 30 us; skew 500 ppm
Sep 23 23:14:45 dist-test-slave-754n krb5kdc[31528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205285, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.227.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:14:45.454494 32231 init.cc:377] Logged in from keytab as oryx/127.24.227.252@KRBTEST.COM (short username oryx)
I20260923 23:14:45.454926 32231 webserver.cc:460] Webserver started at http://127.24.227.252:45531/ using document root <none> and password file <none>
I20260923 23:14:45.455148 32231 fs_manager.cc:359] Metadata directory not provided
I20260923 23:14:45.455214 32231 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:14:45.455361 32231 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:14:45.456379 32231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/master-2/data/instance:
uuid: "f014784de6dd46128a093784ebbe68fb"
format_stamp: "Formatted at 2026-09-23 23:14:45 on dist-test-slave-754n"
server_key: "30cca7dbc2aeed02304e4a151af3294b"
server_key_iv: "c419a283de88cab3c59f477e1a909e6b"
server_key_version: "encryptionkey@0"
I20260923 23:14:45.456699 32231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/master-2/wal/instance:
uuid: "f014784de6dd46128a093784ebbe68fb"
format_stamp: "Formatted at 2026-09-23 23:14:45 on dist-test-slave-754n"
server_key: "30cca7dbc2aeed02304e4a151af3294b"
server_key_iv: "c419a283de88cab3c59f477e1a909e6b"
server_key_version: "encryptionkey@0"
I20260923 23:14:45.458292 32231 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 23:14:45.458923 32246 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:45.459095 32231 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 23:14:45.459178 32231 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/master-2/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/master-2/wal
uuid: "f014784de6dd46128a093784ebbe68fb"
format_stamp: "Formatted at 2026-09-23 23:14:45 on dist-test-slave-754n"
server_key: "30cca7dbc2aeed02304e4a151af3294b"
server_key_iv: "c419a283de88cab3c59f477e1a909e6b"
server_key_version: "encryptionkey@0"
I20260923 23:14:45.459256 32231 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:14:45.487561 32231 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:14:45.489266 32231 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:14:45.489405 32231 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:14:45.492998 32231 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.252:46859
I20260923 23:14:45.493028 32298 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.252:46859 every 8 connection(s)
I20260923 23:14:45.493345 32231 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/master-2/data/info.pb
I20260923 23:14:45.494230 32299 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:14:45.495606 32299 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41951 } has no permanent_uuid. Determining permanent_uuid...
I20260923 23:14:45.500376 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 32231
Sep 23 23:14:45 dist-test-slave-754n krb5kdc[31528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205285, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.227.252@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM
I20260923 23:14:45.504808 31613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:45.496419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35682 (local address 127.24.227.254:41951)
0923 23:14:45.496484 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:45.496486 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:45.496556 (+ 70us) server_negotiation.cc:407] Connection header received
0923 23:14:45.497345 (+ 789us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:45.497347 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:45.497355 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:45.497412 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:45.497931 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:45.498397 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:45.499263 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:45.499432 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:45.499478 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:45.501630 (+ 2152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:45.501631 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:45.501631 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:45.501639 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:45.503193 (+ 1554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:45.503547 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:45.503549 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:45.503549 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:45.503617 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:45.503928 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:45.503929 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:45.503929 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:45.504088 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:45.504194 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:45.504569 (+ 375us) server_negotiation.cc:299] Negotiation successful
0923 23:14:45.504670 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260923 23:14:45.504897 32301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:45.496262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41951 (local address 127.0.0.1:35682)
0923 23:14:45.496439 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:45.496479 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:45.497264 (+ 785us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:45.497480 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:45.497483 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:45.497697 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:14:45.497864 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:45.497867 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:45.498471 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:45.498472 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:45.499194 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:45.499195 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:45.499520 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:45.499521 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:45.499605 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:45.500095 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:14:45.500102 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:14:45.501546 (+ 1444us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:14:45.503269 (+ 1723us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:45.503273 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:45.503275 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:45.503472 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:45.503690 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:45.503690 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:45.503691 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:45.503843 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:45.504185 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:14:45.504192 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:14:45.504374 (+ 182us) client_negotiation.cc:769] Sending connection context
0923 23:14:45.504519 (+ 145us) client_negotiation.cc:240] Negotiation successful
0923 23:14:45.504673 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":84,"threads_started":1}
I20260923 23:14:45.505589 32299 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.227.253" port: 35161 } has no permanent_uuid. Determining permanent_uuid...
Sep 23 23:14:45 dist-test-slave-754n krb5kdc[31528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205284, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.227.252@KRBTEST.COM
Sep 23 23:14:45 dist-test-slave-754n krb5kdc[31528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205285, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.227.252@KRBTEST.COM for oryx/127.24.227.253@KRBTEST.COM
I20260923 23:14:45.511068 32302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:45.500862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41486 (local address 127.24.227.252:46859)
0923 23:14:45.501015 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:45.501019 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:45.501040 (+ 21us) server_negotiation.cc:407] Connection header received
0923 23:14:45.501059 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:45.501061 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:45.501078 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:45.501151 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:45.501710 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:45.502634 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:45.503848 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:45.505094 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:45.505202 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:45.507972 (+ 2770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:45.507978 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:45.507979 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:45.507992 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:45.509677 (+ 1685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:45.510287 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:45.510289 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:45.510289 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:45.510327 (+ 38us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:45.510526 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:45.510527 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:45.510528 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:45.510684 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:45.510814 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:45.510907 (+ 93us) server_negotiation.cc:299] Negotiation successful
0923 23:14:45.510959 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":76,"threads_started":1}
I20260923 23:14:45.512784 32301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:45.505806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.253:35161 (local address 127.0.0.1:50636)
0923 23:14:45.505853 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:45.505863 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:45.505900 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:45.506102 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:45.506103 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:45.506218 (+ 115us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:14:45.506259 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:45.506260 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:45.507199 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:45.507200 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:45.508010 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:45.508012 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:45.508464 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:45.508465 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:45.508527 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:45.508969 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:14:45.508974 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:14:45.510218 (+ 1244us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:14:45.512018 (+ 1800us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:45.512019 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:45.512019 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:45.512217 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:45.512360 (+ 143us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:45.512360 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:45.512360 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:45.512401 (+ 41us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:45.512589 (+ 188us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:14:45.512589 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:14:45.512630 (+ 41us) client_negotiation.cc:769] Sending connection context
0923 23:14:45.512676 (+ 46us) client_negotiation.cc:240] Negotiation successful
0923 23:14:45.512734 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260923 23:14:45.512806 32228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:45.505866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50636 (local address 127.24.227.253:35161)
0923 23:14:45.505924 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:45.505926 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:45.505949 (+ 23us) server_negotiation.cc:407] Connection header received
0923 23:14:45.506003 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:45.506004 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:45.506019 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:45.506075 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:45.506360 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:45.507102 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:45.508079 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:45.508390 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:45.508467 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:45.510284 (+ 1817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:45.510285 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:45.510285 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:45.510291 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:45.511951 (+ 1660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:45.512271 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:45.512272 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:45.512272 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:45.512320 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:45.512432 (+ 112us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:45.512433 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:45.512433 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:45.512529 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:45.512590 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:45.512693 (+ 103us) server_negotiation.cc:299] Negotiation successful
0923 23:14:45.512751 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260923 23:14:45.513293 32299 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.227.252" port: 46859 } has no permanent_uuid. Determining permanent_uuid...
Sep 23 23:14:45 dist-test-slave-754n krb5kdc[31528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205285, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.227.252@KRBTEST.COM for oryx/127.24.227.252@KRBTEST.COM
I20260923 23:14:45.518630 32301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:45.513505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.252:46859 (local address 127.0.0.1:41498)
0923 23:14:45.513586 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:45.513595 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:45.513639 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:45.513716 (+ 77us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:45.513717 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:45.513812 (+ 95us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:14:45.513834 (+ 22us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:45.513835 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:45.514377 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:45.514378 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:45.514812 (+ 434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:45.514813 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:45.515116 (+ 303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:45.515117 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:45.515149 (+ 32us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:45.515440 (+ 291us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:14:45.515444 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:14:45.516485 (+ 1041us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:14:45.517822 (+ 1337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:45.517822 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:45.517822 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:45.518021 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:45.518177 (+ 156us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:45.518178 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:45.518178 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:45.518224 (+ 46us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:45.518376 (+ 152us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:14:45.518377 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:14:45.518461 (+ 84us) client_negotiation.cc:769] Sending connection context
0923 23:14:45.518514 (+ 53us) client_negotiation.cc:240] Negotiation successful
0923 23:14:45.518574 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":70}
I20260923 23:14:45.518682 32302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:45.513554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41498 (local address 127.24.227.252:46859)
0923 23:14:45.513581 (+ 27us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:45.513582 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:45.513629 (+ 47us) server_negotiation.cc:407] Connection header received
0923 23:14:45.513672 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:45.513673 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:45.513679 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:45.513714 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:45.513869 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:45.514305 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:45.514877 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:45.515057 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:45.515104 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:45.516560 (+ 1456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:45.516561 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:45.516561 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:45.516567 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:45.517762 (+ 1195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:45.518084 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:45.518084 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:45.518085 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:45.518128 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:45.518259 (+ 131us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:45.518260 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:45.518260 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:45.518331 (+ 71us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:45.518371 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:45.518533 (+ 162us) server_negotiation.cc:299] Negotiation successful
0923 23:14:45.518613 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":28,"server-negotiator.queue_time_us":13}
I20260923 23:14:45.519774 32299 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f014784de6dd46128a093784ebbe68fb: Bootstrap starting.
I20260923 23:14:45.520550 32299 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f014784de6dd46128a093784ebbe68fb: Neither blocks nor log segments found. Creating new log.
I20260923 23:14:45.520809 32299 log.cc:824] T 00000000000000000000000000000000 P f014784de6dd46128a093784ebbe68fb: Log is configured to *not* fsync() on all Append() calls
I20260923 23:14:45.521474 32299 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f014784de6dd46128a093784ebbe68fb: No bootstrap required, opened a new log
I20260923 23:14:45.522809 32299 raft_consensus.cc:348] T 00000000000000000000000000000000 P f014784de6dd46128a093784ebbe68fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ddae489e6844395a47c75343be04fa5" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41951 } } peers { permanent_uuid: "bd675df8e6274c688bb00c0f39e018c1" member_type: VOTER last_known_addr { host: "127.24.227.253" port: 35161 } } peers { permanent_uuid: "f014784de6dd46128a093784ebbe68fb" member_type: VOTER last_known_addr { host: "127.24.227.252" port: 46859 } }
I20260923 23:14:45.522943 32299 raft_consensus.cc:374] T 00000000000000000000000000000000 P f014784de6dd46128a093784ebbe68fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:14:45.522989 32299 raft_consensus.cc:729] T 00000000000000000000000000000000 P f014784de6dd46128a093784ebbe68fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f014784de6dd46128a093784ebbe68fb, State: Initialized, Role: FOLLOWER
I20260923 23:14:45.523113 32299 consensus_queue.cc:260] T 00000000000000000000000000000000 P f014784de6dd46128a093784ebbe68fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ddae489e6844395a47c75343be04fa5" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41951 } } peers { permanent_uuid: "bd675df8e6274c688bb00c0f39e018c1" member_type: VOTER last_known_addr { host: "127.24.227.253" port: 35161 } } peers { permanent_uuid: "f014784de6dd46128a093784ebbe68fb" member_type: VOTER last_known_addr { host: "127.24.227.252" port: 46859 } }
I20260923 23:14:45.523375 32299 sys_catalog.cc:564] T 00000000000000000000000000000000 P f014784de6dd46128a093784ebbe68fb [sys.catalog]: configured and running, proceeding with master startup.
I20260923 23:14:45.523346 32304 sys_catalog.cc:455] T 00000000000000000000000000000000 P f014784de6dd46128a093784ebbe68fb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ddae489e6844395a47c75343be04fa5" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41951 } } peers { permanent_uuid: "bd675df8e6274c688bb00c0f39e018c1" member_type: VOTER last_known_addr { host: "127.24.227.253" port: 35161 } } peers { permanent_uuid: "f014784de6dd46128a093784ebbe68fb" member_type: VOTER last_known_addr { host: "127.24.227.252" port: 46859 } } }
I20260923 23:14:45.523465 32304 sys_catalog.cc:458] T 00000000000000000000000000000000 P f014784de6dd46128a093784ebbe68fb [sys.catalog]: This master's current role is: FOLLOWER
W20260923 23:14:45.525081 32315 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f014784de6dd46128a093784ebbe68fb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 23:14:45.525141 32315 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 23:14:45.527128 32231 master_runner.cc:428] Detected that this master f014784de6dd46128a093784ebbe68fb is joining an existing cluster
I20260923 23:14:45.527161 32231 master_runner.cc:432] Initiating AddMaster RPC to add 127.24.227.252:46859
I20260923 23:14:45.533761 32322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:45.528004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41951 (local address 127.0.0.1:35692)
0923 23:14:45.528154 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:45.528163 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:45.528229 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:45.528434 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:45.528435 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:45.528639 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:14:45.528703 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:45.528705 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:45.529276 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:45.529277 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:45.529948 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:45.529949 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:45.530403 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:45.530404 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:45.530454 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:45.530814 (+ 360us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:14:45.530818 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:14:45.531060 (+ 242us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:14:45.532784 (+ 1724us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:45.532785 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:45.532785 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:45.532953 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:45.533141 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:45.533141 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:45.533141 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:45.533197 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:45.533459 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:14:45.533460 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:14:45.533509 (+ 49us) client_negotiation.cc:769] Sending connection context
0923 23:14:45.533576 (+ 67us) client_negotiation.cc:240] Negotiation successful
0923 23:14:45.533660 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":69,"threads_started":1}
I20260923 23:14:45.533778 31613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:45.528041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35692 (local address 127.24.227.254:41951)
0923 23:14:45.528079 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:45.528081 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:45.528207 (+ 126us) server_negotiation.cc:407] Connection header received
0923 23:14:45.528330 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:45.528331 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:45.528342 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:45.528389 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:45.528782 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:45.529197 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:45.530065 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:45.530321 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:45.530385 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:45.531136 (+ 751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:45.531137 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:45.531137 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:45.531143 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:45.532713 (+ 1570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:45.533041 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:45.533042 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:45.533043 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:45.533094 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:45.533257 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:45.533258 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:45.533258 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:45.533389 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:45.533450 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:45.533610 (+ 160us) server_negotiation.cc:299] Negotiation successful
0923 23:14:45.533674 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
Sep 23 23:14:45 dist-test-slave-754n krb5kdc[31528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205284, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.227.254@KRBTEST.COM for oryx/127.24.227.252@KRBTEST.COM
I20260923 23:14:45.547730 32302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:45.528397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41514 (local address 127.24.227.252:46859)
0923 23:14:45.528424 (+ 27us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:45.528426 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:45.532015 (+ 3589us) server_negotiation.cc:407] Connection header received
0923 23:14:45.532038 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:45.532039 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:45.532048 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:45.532111 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:45.533089 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:45.533690 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:45.535051 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:45.535334 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:45.535404 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:45.545154 (+ 9750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:45.545156 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:45.545156 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:45.545164 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:45.546556 (+ 1392us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:45.547039 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:45.547040 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:45.547040 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:45.547084 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:45.547246 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:45.547246 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:45.547247 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:45.547343 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:45.547378 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:45.547561 (+ 183us) server_negotiation.cc:299] Negotiation successful
0923 23:14:45.547652 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13}
I20260923 23:14:45.547977 32228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:45.528372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50648 (local address 127.24.227.253:35161)
0923 23:14:45.528414 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:45.528416 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:45.528423 (+ 7us) server_negotiation.cc:407] Connection header received
0923 23:14:45.528438 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:45.528439 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:45.528448 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:45.528510 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:45.528839 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:45.529451 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:45.530208 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:45.530420 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:45.530476 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:45.544996 (+ 14520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:45.545006 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:45.545006 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:45.545013 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:45.546466 (+ 1453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:45.547078 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:45.547079 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:45.547080 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:45.547134 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:45.547331 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:45.547332 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:45.547332 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:45.547453 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:45.547492 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:45.547700 (+ 208us) server_negotiation.cc:299] Negotiation successful
0923 23:14:45.547780 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260923 23:14:45.548216 32323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:45.528243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.252:46859 (local address 127.0.0.1:41514)
0923 23:14:45.531925 (+ 3682us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:45.531935 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:45.531987 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:45.532114 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:45.532115 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:45.532975 (+ 860us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:14:45.533036 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:45.533038 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:45.533762 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:45.533762 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:45.534970 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:45.534973 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:45.535417 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:45.535418 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:45.535497 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:45.544485 (+ 8988us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:14:45.544490 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:14:45.545061 (+ 571us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:14:45.546633 (+ 1572us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:45.546634 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:45.546634 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:45.546950 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:45.547144 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:45.547144 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:45.547145 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:45.547188 (+ 43us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:45.547414 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:14:45.547414 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:14:45.547443 (+ 29us) client_negotiation.cc:769] Sending connection context
0923 23:14:45.548022 (+ 579us) client_negotiation.cc:240] Negotiation successful
0923 23:14:45.548102 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3667,"thread_start_us":67,"threads_started":1}
I20260923 23:14:45.548625 32131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:45.536137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.252:46859 (local address 127.0.0.1:41524)
0923 23:14:45.536193 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:45.536203 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:45.536243 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:45.536405 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:45.536405 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:45.536550 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:14:45.536585 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:45.536586 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:45.537111 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:45.537112 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:45.537857 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:45.537859 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:45.538201 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:45.538202 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:45.538269 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:45.538604 (+ 335us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:14:45.538608 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:14:45.539733 (+ 1125us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:14:45.547537 (+ 7804us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:45.547538 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:45.547539 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:45.547742 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:45.548013 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:45.548014 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:45.548014 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:45.548070 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:45.548342 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:14:45.548343 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:14:45.548397 (+ 54us) client_negotiation.cc:769] Sending connection context
0923 23:14:45.548441 (+ 44us) client_negotiation.cc:240] Negotiation successful
0923 23:14:45.548543 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260923 23:14:45.548681 32321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:45.528013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.253:35161 (local address 127.0.0.1:50648)
0923 23:14:45.528158 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:45.528167 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:45.528209 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:45.528521 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:45.528522 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:45.528667 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:14:45.528713 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:45.528714 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:45.529530 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:45.529530 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:45.530141 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:45.530142 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:45.530487 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:45.530488 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:45.530545 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:45.544414 (+ 13869us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:14:45.544419 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:14:45.544896 (+ 477us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:14:45.546548 (+ 1652us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:45.546549 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:45.546549 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:45.546978 (+ 429us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:45.547207 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:45.547208 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:45.547208 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:45.547263 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:45.547537 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:14:45.547537 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:14:45.547578 (+ 41us) client_negotiation.cc:769] Sending connection context
0923 23:14:45.548490 (+ 912us) client_negotiation.cc:240] Negotiation successful
0923 23:14:45.548573 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":89,"threads_started":1}
I20260923 23:14:45.549286 32324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:45.536167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41524 (local address 127.24.227.252:46859)
0923 23:14:45.536282 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:45.536283 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:45.536290 (+ 7us) server_negotiation.cc:407] Connection header received
0923 23:14:45.536331 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:45.536331 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:45.536338 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:45.536386 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:45.536656 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:45.537045 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:45.537940 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:45.538138 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:45.538180 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:45.539828 (+ 1648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:45.539830 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:45.539830 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:45.539836 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:45.547436 (+ 7600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:45.547817 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:45.547818 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:45.547818 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:45.547927 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:45.548145 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:45.548146 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:45.548146 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:45.548267 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:45.549111 (+ 844us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:45.549147 (+ 36us) server_negotiation.cc:299] Negotiation successful
0923 23:14:45.549218 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":68,"threads_started":1}
I20260923 23:14:45.549654 31572 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.24.227.252:46859: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "f014784de6dd46128a093784ebbe68fb"
member_type: NON_VOTER
last_known_addr {
host: "127.24.227.252"
port: 46859
}
attrs {
promote: true
}
}
dest_uuid: "2ddae489e6844395a47c75343be04fa5"
cas_config_opid_index: 5
I20260923 23:14:45.549877 31572 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2ddae489e6844395a47c75343be04fa5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "2ddae489e6844395a47c75343be04fa5" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41951 } } peers { permanent_uuid: "bd675df8e6274c688bb00c0f39e018c1" member_type: NON_VOTER last_known_addr { host: "127.24.227.253" port: 35161 } attrs { promote: true } } peers { permanent_uuid: "f014784de6dd46128a093784ebbe68fb" member_type: NON_VOTER last_known_addr { host: "127.24.227.252" port: 46859 } attrs { promote: true } }
I20260923 23:14:45.550217 32133 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 2ddae489e6844395a47c75343be04fa5 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER f014784de6dd46128a093784ebbe68fb (127.24.227.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "2ddae489e6844395a47c75343be04fa5" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41951 } } peers { permanent_uuid: "bd675df8e6274c688bb00c0f39e018c1" member_type: NON_VOTER last_known_addr { host: "127.24.227.253" port: 35161 } attrs { promote: true } } peers { permanent_uuid: "f014784de6dd46128a093784ebbe68fb" member_type: NON_VOTER last_known_addr { host: "127.24.227.252" port: 46859 } attrs { promote: true } } }
I20260923 23:14:45.550660 32273 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f014784de6dd46128a093784ebbe68fb [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:14:45.551189 32133 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.24.227.252:46859
I20260923 23:14:45.551256 32133 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ddae489e6844395a47c75343be04fa5 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "2ddae489e6844395a47c75343be04fa5" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "2ddae489e6844395a47c75343be04fa5" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41951 } } peers { permanent_uuid: "bd675df8e6274c688bb00c0f39e018c1" member_type: NON_VOTER last_known_addr { host: "127.24.227.253" port: 35161 } attrs { promote: true } } peers { permanent_uuid: "f014784de6dd46128a093784ebbe68fb" member_type: NON_VOTER last_known_addr { host: "127.24.227.252" port: 46859 } attrs { promote: true } } }
I20260923 23:14:45.551314 32133 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ddae489e6844395a47c75343be04fa5 [sys.catalog]: This master's current role is: LEADER
I20260923 23:14:45.551489 32231 master.cc:537] Master@127.24.227.252:46859 shutting down...
I20260923 23:14:45.551517 32273 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f014784de6dd46128a093784ebbe68fb [term 1 FOLLOWER]: Refusing update from remote peer 2ddae489e6844395a47c75343be04fa5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260923 23:14:45.551797 32132 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2ddae489e6844395a47c75343be04fa5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f014784de6dd46128a093784ebbe68fb" member_type: NON_VOTER last_known_addr { host: "127.24.227.252" port: 46859 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260923 23:14:45.552313 31559 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260923 23:14:45.552753 31559 consensus_peers.cc:487] T 00000000000000000000000000000000 P 2ddae489e6844395a47c75343be04fa5 -> Peer f014784de6dd46128a093784ebbe68fb (127.24.227.252:46859): Couldn't send request to peer f014784de6dd46128a093784ebbe68fb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260923 23:14:45.554435 32231 raft_consensus.cc:2229] T 00000000000000000000000000000000 P f014784de6dd46128a093784ebbe68fb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 23:14:45.554517 32231 raft_consensus.cc:2258] T 00000000000000000000000000000000 P f014784de6dd46128a093784ebbe68fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 23:14:45.554564 32231 tablet_replica.cc:330] T 00000000000000000000000000000000 P f014784de6dd46128a093784ebbe68fb: stopping tablet replica
I20260923 23:14:45.554908 32131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:45.550238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.253:35161 (local address 127.0.0.1:50654)
0923 23:14:45.551788 (+ 1550us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:45.551797 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:45.551835 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:45.552058 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:45.552059 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:45.552060 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:14:45.552090 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:45.552091 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:45.552831 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:45.552832 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:45.553756 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:45.553758 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:45.554648 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:45.554649 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:45.554718 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:45.554739 (+ 21us) client_negotiation.cc:769] Sending connection context
0923 23:14:45.554812 (+ 73us) client_negotiation.cc:240] Negotiation successful
0923 23:14:45.554848 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1537}
I20260923 23:14:45.555105 32228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:45.551657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50654 (local address 127.24.227.253:35161)
0923 23:14:45.551691 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:45.551693 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:45.551902 (+ 209us) server_negotiation.cc:407] Connection header received
0923 23:14:45.551922 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:45.551923 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:45.551945 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:45.551980 (+ 35us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:14:45.552157 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:45.552757 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:45.553852 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:45.554470 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:45.554550 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:45.554616 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:45.555013 (+ 397us) server_negotiation.cc:299] Negotiation successful
0923 23:14:45.555052 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260923 23:14:45.555408 32187 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bd675df8e6274c688bb00c0f39e018c1 [term 1 LEARNER]: Refusing update from remote peer 2ddae489e6844395a47c75343be04fa5: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260923 23:14:45.555650 32132 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2ddae489e6844395a47c75343be04fa5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd675df8e6274c688bb00c0f39e018c1" member_type: NON_VOTER last_known_addr { host: "127.24.227.253" port: 35161 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260923 23:14:45.556077 32187 raft_consensus.cc:2932] T 00000000000000000000000000000000 P bd675df8e6274c688bb00c0f39e018c1 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER f014784de6dd46128a093784ebbe68fb (127.24.227.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "2ddae489e6844395a47c75343be04fa5" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41951 } } peers { permanent_uuid: "bd675df8e6274c688bb00c0f39e018c1" member_type: NON_VOTER last_known_addr { host: "127.24.227.253" port: 35161 } attrs { promote: true } } peers { permanent_uuid: "f014784de6dd46128a093784ebbe68fb" member_type: NON_VOTER last_known_addr { host: "127.24.227.252" port: 46859 } attrs { promote: true } } }
I20260923 23:14:45.557080 32216 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd675df8e6274c688bb00c0f39e018c1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ddae489e6844395a47c75343be04fa5. Latest consensus state: current_term: 1 leader_uuid: "2ddae489e6844395a47c75343be04fa5" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "2ddae489e6844395a47c75343be04fa5" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41951 } } peers { permanent_uuid: "bd675df8e6274c688bb00c0f39e018c1" member_type: NON_VOTER last_known_addr { host: "127.24.227.253" port: 35161 } attrs { promote: true } } peers { permanent_uuid: "f014784de6dd46128a093784ebbe68fb" member_type: NON_VOTER last_known_addr { host: "127.24.227.252" port: 46859 } attrs { promote: true } } }
I20260923 23:14:45.557175 32216 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd675df8e6274c688bb00c0f39e018c1 [sys.catalog]: This master's current role is: LEARNER
I20260923 23:14:45.557190 32132 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 2ddae489e6844395a47c75343be04fa5: attempting to promote NON_VOTER bd675df8e6274c688bb00c0f39e018c1 to VOTER
I20260923 23:14:45.557212 32216 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd675df8e6274c688bb00c0f39e018c1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2ddae489e6844395a47c75343be04fa5" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "2ddae489e6844395a47c75343be04fa5" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41951 } } peers { permanent_uuid: "bd675df8e6274c688bb00c0f39e018c1" member_type: NON_VOTER last_known_addr { host: "127.24.227.253" port: 35161 } attrs { promote: true } } peers { permanent_uuid: "f014784de6dd46128a093784ebbe68fb" member_type: NON_VOTER last_known_addr { host: "127.24.227.252" port: 46859 } attrs { promote: true } } }
I20260923 23:14:45.557250 32216 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd675df8e6274c688bb00c0f39e018c1 [sys.catalog]: This master's current role is: LEARNER
I20260923 23:14:45.557361 32132 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2ddae489e6844395a47c75343be04fa5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2ddae489e6844395a47c75343be04fa5" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41951 } } peers { permanent_uuid: "bd675df8e6274c688bb00c0f39e018c1" member_type: VOTER last_known_addr { host: "127.24.227.253" port: 35161 } attrs { promote: false } } peers { permanent_uuid: "f014784de6dd46128a093784ebbe68fb" member_type: NON_VOTER last_known_addr { host: "127.24.227.252" port: 46859 } attrs { promote: true } }
I20260923 23:14:45.557885 32131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:45.557768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.252:46859 (local address 127.0.0.1:41534)
0923 23:14:45.557817 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:45.557860 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.252:46859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260923 23:14:45.557991 32187 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bd675df8e6274c688bb00c0f39e018c1 [term 1 LEARNER]: Refusing update from remote peer 2ddae489e6844395a47c75343be04fa5: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260923 23:14:45.558161 32131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:45.558085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.252:46859 (local address 127.0.0.1:41546)
0923 23:14:45.558117 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:45.558139 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.252:46859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260923 23:14:45.558220 31559 consensus_peers.cc:487] T 00000000000000000000000000000000 P 2ddae489e6844395a47c75343be04fa5 -> Peer f014784de6dd46128a093784ebbe68fb (127.24.227.252:46859): Couldn't send request to peer f014784de6dd46128a093784ebbe68fb. Status: Network error: Client connection negotiation failed: client connection to 127.24.227.252:46859: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260923 23:14:45.558229 32133 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2ddae489e6844395a47c75343be04fa5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd675df8e6274c688bb00c0f39e018c1" member_type: VOTER last_known_addr { host: "127.24.227.253" port: 35161 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260923 23:14:45.558805 32328 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 2ddae489e6844395a47c75343be04fa5 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, bd675df8e6274c688bb00c0f39e018c1 (127.24.227.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2ddae489e6844395a47c75343be04fa5" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41951 } } peers { permanent_uuid: "bd675df8e6274c688bb00c0f39e018c1" member_type: VOTER last_known_addr { host: "127.24.227.253" port: 35161 } attrs { promote: false } } peers { permanent_uuid: "f014784de6dd46128a093784ebbe68fb" member_type: NON_VOTER last_known_addr { host: "127.24.227.252" port: 46859 } attrs { promote: true } } }
I20260923 23:14:45.559005 32187 raft_consensus.cc:2932] T 00000000000000000000000000000000 P bd675df8e6274c688bb00c0f39e018c1 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, bd675df8e6274c688bb00c0f39e018c1 (127.24.227.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2ddae489e6844395a47c75343be04fa5" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41951 } } peers { permanent_uuid: "bd675df8e6274c688bb00c0f39e018c1" member_type: VOTER last_known_addr { host: "127.24.227.253" port: 35161 } attrs { promote: false } } peers { permanent_uuid: "f014784de6dd46128a093784ebbe68fb" member_type: NON_VOTER last_known_addr { host: "127.24.227.252" port: 46859 } attrs { promote: true } } }
I20260923 23:14:45.559700 32328 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ddae489e6844395a47c75343be04fa5 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "2ddae489e6844395a47c75343be04fa5" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2ddae489e6844395a47c75343be04fa5" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41951 } } peers { permanent_uuid: "bd675df8e6274c688bb00c0f39e018c1" member_type: VOTER last_known_addr { host: "127.24.227.253" port: 35161 } attrs { promote: false } } peers { permanent_uuid: "f014784de6dd46128a093784ebbe68fb" member_type: NON_VOTER last_known_addr { host: "127.24.227.252" port: 46859 } attrs { promote: true } } }
I20260923 23:14:45.559773 32328 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ddae489e6844395a47c75343be04fa5 [sys.catalog]: This master's current role is: LEADER
I20260923 23:14:45.559826 32328 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ddae489e6844395a47c75343be04fa5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2ddae489e6844395a47c75343be04fa5" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2ddae489e6844395a47c75343be04fa5" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41951 } } peers { permanent_uuid: "bd675df8e6274c688bb00c0f39e018c1" member_type: VOTER last_known_addr { host: "127.24.227.253" port: 35161 } attrs { promote: false } } peers { permanent_uuid: "f014784de6dd46128a093784ebbe68fb" member_type: NON_VOTER last_known_addr { host: "127.24.227.252" port: 46859 } attrs { promote: true } } }
I20260923 23:14:45.559916 32328 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ddae489e6844395a47c75343be04fa5 [sys.catalog]: This master's current role is: LEADER
I20260923 23:14:45.559921 32327 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd675df8e6274c688bb00c0f39e018c1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2ddae489e6844395a47c75343be04fa5" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2ddae489e6844395a47c75343be04fa5" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41951 } } peers { permanent_uuid: "bd675df8e6274c688bb00c0f39e018c1" member_type: VOTER last_known_addr { host: "127.24.227.253" port: 35161 } attrs { promote: false } } peers { permanent_uuid: "f014784de6dd46128a093784ebbe68fb" member_type: NON_VOTER last_known_addr { host: "127.24.227.252" port: 46859 } attrs { promote: true } } }
I20260923 23:14:45.560019 32327 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd675df8e6274c688bb00c0f39e018c1 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 23:14:45.568720 32231 master.cc:552] Master@127.24.227.252:46859 shutdown complete.
I20260923 23:14:45.568764 32231 master_runner.cc:305] Clearing existing system tablet
I20260923 23:14:45.568878 32231 fs_manager.cc:359] Metadata directory not provided
I20260923 23:14:45.568920 32231 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:14:45.569523 32231 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260923 23:14:45.569947 32330 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:45.570080 32231 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:14:45.570132 32231 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:45.570154 32231 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/master-2/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/master-2/wal
uuid: "f014784de6dd46128a093784ebbe68fb"
format_stamp: "Formatted at 2026-09-23 23:14:45 on dist-test-slave-754n"
server_key: "30cca7dbc2aeed02304e4a151af3294b"
server_key_iv: "c419a283de88cab3c59f477e1a909e6b"
server_key_version: "encryptionkey@0"
I20260923 23:14:45.570793 32231 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P f014784de6dd46128a093784ebbe68fb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 23:14:45.572225 32231 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P f014784de6dd46128a093784ebbe68fb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260923 23:14:45.572292 32231 log.cc:1196] T 00000000000000000000000000000000 P f014784de6dd46128a093784ebbe68fb: Deleting WAL directory at /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260923 23:14:45.572475 32231 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P f014784de6dd46128a093784ebbe68fb: Deleting consensus metadata
I20260923 23:14:45.572611 32231 master_runner.cc:315] Copying system tablet from 127.24.227.254:41951
I20260923 23:14:45.573167 32231 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P f014784de6dd46128a093784ebbe68fb: tablet copy: Beginning tablet copy session from remote peer at address 127.24.227.254:41951
I20260923 23:14:45.581525 31613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:45.573462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35696 (local address 127.24.227.254:41951)
0923 23:14:45.573499 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:45.573500 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:45.576040 (+ 2540us) server_negotiation.cc:407] Connection header received
0923 23:14:45.576096 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:45.576096 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:45.576104 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:45.576145 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:45.576437 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:45.576905 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:45.577844 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:45.577999 (+ 155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:45.578043 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:45.578946 (+ 903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:45.578947 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:45.578947 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:45.578953 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:45.580439 (+ 1486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:45.580811 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:45.580812 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:45.580813 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:45.580886 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:45.581088 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:45.581089 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:45.581089 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:45.581206 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:45.581264 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:45.581418 (+ 154us) server_negotiation.cc:299] Negotiation successful
0923 23:14:45.581469 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 23:14:45.581573 32336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:45.573401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41951 (local address 127.0.0.1:35696)
0923 23:14:45.575960 (+ 2559us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:45.575971 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:45.576038 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:45.576158 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:45.576159 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:45.576321 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:14:45.576377 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:45.576379 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:45.576973 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:45.576974 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:45.577776 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:45.577779 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:45.578064 (+ 285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:45.578065 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:45.578125 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:45.578517 (+ 392us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:14:45.578521 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:14:45.578852 (+ 331us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:14:45.580508 (+ 1656us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:45.580509 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:45.580510 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:45.580743 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:45.580951 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:45.580951 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:45.580952 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:45.581031 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:45.581275 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:14:45.581275 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:14:45.581331 (+ 56us) client_negotiation.cc:769] Sending connection context
0923 23:14:45.581395 (+ 64us) client_negotiation.cc:240] Negotiation successful
0923 23:14:45.581480 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2535,"thread_start_us":62,"threads_started":1}
I20260923 23:14:45.582064 31593 tablet_copy_service.cc:140] P 2ddae489e6844395a47c75343be04fa5: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer f014784de6dd46128a093784ebbe68fb ({username='oryx', principal='oryx/127.24.227.252@KRBTEST.COM'} at 127.0.0.1:35696)
I20260923 23:14:45.582122 31593 tablet_copy_service.cc:161] P 2ddae489e6844395a47c75343be04fa5: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer f014784de6dd46128a093784ebbe68fb at {username='oryx', principal='oryx/127.24.227.252@KRBTEST.COM'} at 127.0.0.1:35696: session id = f014784de6dd46128a093784ebbe68fb-00000000000000000000000000000000
I20260923 23:14:45.582643 31593 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 2ddae489e6844395a47c75343be04fa5: Tablet Copy: opened 0 blocks and 1 log segments
I20260923 23:14:45.583081 32231 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260923 23:14:45.584198 32231 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P f014784de6dd46128a093784ebbe68fb: tablet copy: Starting download of 0 data blocks...
I20260923 23:14:45.584328 32231 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P f014784de6dd46128a093784ebbe68fb: tablet copy: Starting download of 1 WAL segments...
I20260923 23:14:45.585086 32231 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P f014784de6dd46128a093784ebbe68fb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260923 23:14:45.585939 31593 tablet_copy_service.cc:342] P 2ddae489e6844395a47c75343be04fa5: Request end of tablet copy session f014784de6dd46128a093784ebbe68fb-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.24.227.252@KRBTEST.COM'} at 127.0.0.1:35696
I20260923 23:14:45.585988 31593 tablet_copy_service.cc:433] P 2ddae489e6844395a47c75343be04fa5: ending tablet copy session f014784de6dd46128a093784ebbe68fb-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer f014784de6dd46128a093784ebbe68fb
I20260923 23:14:45.588060 32231 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:14:45.589140 32338 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:45.589150 32337 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:14:45.589226 32231 server_base.cc:1034] running on GCE node
W20260923 23:14:45.589341 32340 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:14:45.589491 32231 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:14:45.589653 32231 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:14:45.590781 32231 hybrid_clock.cc:648] HybridClock initialized: now 1790205285590771 us; error 24 us; skew 500 ppm
Sep 23 23:14:45 dist-test-slave-754n krb5kdc[31528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205285, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.227.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:14:45.591996 32231 init.cc:377] Logged in from keytab as oryx/127.24.227.252@KRBTEST.COM (short username oryx)
I20260923 23:14:45.592259 32231 webserver.cc:460] Webserver started at http://127.24.227.252:36415/ using document root <none> and password file <none>
I20260923 23:14:45.592360 32231 fs_manager.cc:359] Metadata directory not provided
I20260923 23:14:45.592410 32231 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:14:45.592972 32231 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260923 23:14:45.593430 32347 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:45.593641 32231 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:45.593686 32231 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/master-2/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/master-2/wal
uuid: "f014784de6dd46128a093784ebbe68fb"
format_stamp: "Formatted at 2026-09-23 23:14:45 on dist-test-slave-754n"
server_key: "30cca7dbc2aeed02304e4a151af3294b"
server_key_iv: "c419a283de88cab3c59f477e1a909e6b"
server_key_version: "encryptionkey@0"
I20260923 23:14:45.593737 32231 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:14:45.608520 32231 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:14:45.610023 32231 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:14:45.613061 32231 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.252:46859
I20260923 23:14:45.613076 32399 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.252:46859 every 8 connection(s)
I20260923 23:14:45.613351 32231 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/master-2/data/info.pb
I20260923 23:14:45.614192 32400 sys_catalog.cc:263] Verifying existing consensus state
I20260923 23:14:45.614672 32400 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f014784de6dd46128a093784ebbe68fb: Bootstrap starting.
I20260923 23:14:45.617616 32400 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f014784de6dd46128a093784ebbe68fb: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 23:14:45.617821 32400 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f014784de6dd46128a093784ebbe68fb: Bootstrap complete.
I20260923 23:14:45.617996 32400 raft_consensus.cc:348] T 00000000000000000000000000000000 P f014784de6dd46128a093784ebbe68fb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2ddae489e6844395a47c75343be04fa5" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41951 } } peers { permanent_uuid: "bd675df8e6274c688bb00c0f39e018c1" member_type: VOTER last_known_addr { host: "127.24.227.253" port: 35161 } attrs { promote: false } } peers { permanent_uuid: "f014784de6dd46128a093784ebbe68fb" member_type: NON_VOTER last_known_addr { host: "127.24.227.252" port: 46859 } attrs { promote: true } }
I20260923 23:14:45.618091 32400 raft_consensus.cc:729] T 00000000000000000000000000000000 P f014784de6dd46128a093784ebbe68fb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f014784de6dd46128a093784ebbe68fb, State: Initialized, Role: LEARNER
I20260923 23:14:45.618139 32400 consensus_queue.cc:260] T 00000000000000000000000000000000 P f014784de6dd46128a093784ebbe68fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2ddae489e6844395a47c75343be04fa5" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41951 } } peers { permanent_uuid: "bd675df8e6274c688bb00c0f39e018c1" member_type: VOTER last_known_addr { host: "127.24.227.253" port: 35161 } attrs { promote: false } } peers { permanent_uuid: "f014784de6dd46128a093784ebbe68fb" member_type: NON_VOTER last_known_addr { host: "127.24.227.252" port: 46859 } attrs { promote: true } }
I20260923 23:14:45.618342 32400 sys_catalog.cc:564] T 00000000000000000000000000000000 P f014784de6dd46128a093784ebbe68fb [sys.catalog]: configured and running, proceeding with master startup.
I20260923 23:14:45.618336 32403 sys_catalog.cc:455] T 00000000000000000000000000000000 P f014784de6dd46128a093784ebbe68fb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2ddae489e6844395a47c75343be04fa5" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41951 } } peers { permanent_uuid: "bd675df8e6274c688bb00c0f39e018c1" member_type: VOTER last_known_addr { host: "127.24.227.253" port: 35161 } attrs { promote: false } } peers { permanent_uuid: "f014784de6dd46128a093784ebbe68fb" member_type: NON_VOTER last_known_addr { host: "127.24.227.252" port: 46859 } attrs { promote: true } } }
I20260923 23:14:45.618438 32403 sys_catalog.cc:458] T 00000000000000000000000000000000 P f014784de6dd46128a093784ebbe68fb [sys.catalog]: This master's current role is: LEARNER
I20260923 23:14:45.619753 32414 catalog_manager.cc:1259] Loaded cluster ID: 25a76464fda54de4ba6067e47f542e90
I20260923 23:14:45.619966 32414 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f014784de6dd46128a093784ebbe68fb: loading cluster ID for follower catalog manager: success
I20260923 23:14:45.621035 32414 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f014784de6dd46128a093784ebbe68fb: acquiring CA information for follower catalog manager: success
I20260923 23:14:45.621294 32414 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f014784de6dd46128a093784ebbe68fb: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260923 23:14:45.669505 32415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:45.663379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41574 (local address 127.24.227.252:46859)
0923 23:14:45.663525 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:45.663527 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:45.663535 (+ 8us) server_negotiation.cc:407] Connection header received
0923 23:14:45.663565 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:45.663566 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:45.663574 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:45.663672 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:45.664262 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:45.664910 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:45.665637 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:45.665848 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:45.665940 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:45.667098 (+ 1158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:45.667099 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:45.667099 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:45.667106 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:45.668371 (+ 1265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:45.668726 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:45.668727 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:45.668727 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:45.668782 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:45.668971 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:45.668972 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:45.668972 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:45.669078 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:45.669173 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:45.669317 (+ 144us) server_negotiation.cc:299] Negotiation successful
0923 23:14:45.669397 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":58,"threads_started":1}
I20260923 23:14:45.670044 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 31635
I20260923 23:14:45.674546 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.227.193:42775
--local_ip_for_outbound_sockets=127.24.227.193
--tserver_master_addrs=127.24.227.254:41951
--webserver_port=46723
--webserver_interface=127.24.227.193
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/krb5kdc/oryx_127.24.227.193.keytab
--principal=oryx/127.24.227.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:42907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.24.227.254:41951,127.24.227.253:35161
--tserver_master_addrs=127.24.227.254:41951,127.24.227.253:35161,127.24.227.252:46859 with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:14:45.750427 32416 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:14:45.750612 32416 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:14:45.750635 32416 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:14:45.751749 32416 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:14:45.751784 32416 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:14:45.751807 32416 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.227.193
I20260923 23:14:45.753153 32416 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:42907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/krb5kdc/oryx_127.24.227.193.keytab
--principal=oryx/127.24.227.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.227.193:42775
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.227.193
--webserver_port=46723
--tserver_master_addrs=127.24.227.254:41951,127.24.227.253:35161,127.24.227.252:46859
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32416
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.227.193
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:14:45.753333 32416 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:14:45.753485 32416 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:14:45.758250 32421 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:14:45.758395 32416 server_base.cc:1034] running on GCE node
W20260923 23:14:45.758250 32422 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:45.758520 32424 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:14:45.758733 32416 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:14:45.758953 32416 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:14:45.760105 32416 hybrid_clock.cc:648] HybridClock initialized: now 1790205285760100 us; error 37 us; skew 500 ppm
Sep 23 23:14:45 dist-test-slave-754n krb5kdc[31528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205285, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.227.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:14:45.762369 32416 init.cc:377] Logged in from keytab as oryx/127.24.227.193@KRBTEST.COM (short username oryx)
I20260923 23:14:45.762809 32416 webserver.cc:460] Webserver started at http://127.24.227.193:46723/ using document root <none> and password file <none>
I20260923 23:14:45.762974 32416 fs_manager.cc:359] Metadata directory not provided
I20260923 23:14:45.763023 32416 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:14:45.764256 32416 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 23:14:45.764765 32431 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:45.764943 32416 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:45.765005 32416 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/ts-0/wal
uuid: "85902a6195e544b7a8e8794e0b7c7ca1"
format_stamp: "Formatted at 2026-09-23 23:14:44 on dist-test-slave-754n"
server_key: "c30d4b74d78050e9c47610f2306a99cc"
server_key_iv: "cb4136723a2eb13f0ea8688c9b9822a0"
server_key_version: "encryptionkey@0"
I20260923 23:14:45.765275 32416 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:14:45.775681 32416 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:14:45.777261 32416 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:14:45.777359 32416 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:14:45.777505 32416 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 23:14:45.777714 32416 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 23:14:45.777741 32416 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:45.777760 32416 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 23:14:45.777772 32416 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:45.783815 32416 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.193:42775
I20260923 23:14:45.783875 32544 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.193:42775 every 8 connection(s)
I20260923 23:14:45.784301 32416 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/ts-0/data/info.pb
I20260923 23:14:45.788667 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 32416
I20260923 23:14:45.788818 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 31769
Sep 23 23:14:45 dist-test-slave-754n krb5kdc[31528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205285, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.227.193@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM
Sep 23 23:14:45 dist-test-slave-754n krb5kdc[31528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205285, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.227.193@KRBTEST.COM for oryx/127.24.227.253@KRBTEST.COM
Sep 23 23:14:45 dist-test-slave-754n krb5kdc[31528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205285, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.227.193@KRBTEST.COM for oryx/127.24.227.252@KRBTEST.COM
I20260923 23:14:45.795823 31613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:45.785273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.193:42519 (local address 127.24.227.254:41951)
0923 23:14:45.785343 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:45.785346 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:45.785357 (+ 11us) server_negotiation.cc:407] Connection header received
0923 23:14:45.786354 (+ 997us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:45.786356 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:45.786367 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:45.786438 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:45.787054 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:45.787702 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:45.789108 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:45.789315 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:45.789352 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:45.792483 (+ 3131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:45.792485 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:45.792485 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:45.792493 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:45.794415 (+ 1922us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:45.794874 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:45.794876 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:45.794877 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:45.794934 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:45.795195 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:45.795196 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:45.795196 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:45.795309 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:45.795381 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:45.795651 (+ 270us) server_negotiation.cc:299] Negotiation successful
0923 23:14:45.795723 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260923 23:14:45.796180 32551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:45.784894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41951 (local address 127.24.227.193:42519)
0923 23:14:45.785138 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:45.785177 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:45.786112 (+ 935us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:45.786458 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:45.786459 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:45.786782 (+ 323us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:14:45.786953 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:45.786957 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:45.787807 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:45.787808 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:45.789008 (+ 1200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:45.789010 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:45.789428 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:45.789428 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:45.789550 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:45.790123 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:14:45.790133 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:14:45.792373 (+ 2240us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:14:45.794506 (+ 2133us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:45.794513 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:45.794515 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:45.794781 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:45.795013 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:45.795014 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:45.795014 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:45.795119 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:45.795373 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:14:45.795379 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:14:45.795510 (+ 131us) client_negotiation.cc:769] Sending connection context
0923 23:14:45.795631 (+ 121us) client_negotiation.cc:240] Negotiation successful
0923 23:14:45.795809 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":208,"mutex_wait_us":2,"thread_start_us":81,"threads_started":1}
I20260923 23:14:45.796416 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.227.194:33365
--local_ip_for_outbound_sockets=127.24.227.194
--tserver_master_addrs=127.24.227.254:41951
--webserver_port=45683
--webserver_interface=127.24.227.194
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/krb5kdc/oryx_127.24.227.194.keytab
--principal=oryx/127.24.227.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:42907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.24.227.254:41951,127.24.227.253:35161
--tserver_master_addrs=127.24.227.254:41951,127.24.227.253:35161,127.24.227.252:46859 with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 23:14:45.796748 32546 heartbeater.cc:347] Connected to a master server at 127.24.227.254:41951
I20260923 23:14:45.796826 32546 heartbeater.cc:464] Registering TS with master...
I20260923 23:14:45.796892 32228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:45.784923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.193:51243 (local address 127.24.227.253:35161)
0923 23:14:45.785196 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:45.785198 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:45.785401 (+ 203us) server_negotiation.cc:407] Connection header received
0923 23:14:45.786218 (+ 817us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:45.786219 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:45.786233 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:45.786295 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:45.787417 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:45.788066 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:45.789109 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:45.789429 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:45.789470 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:45.792474 (+ 3004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:45.792475 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:45.792475 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:45.792482 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:45.794748 (+ 2266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:45.796025 (+ 1277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:45.796026 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:45.796027 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:45.796082 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:45.796282 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:45.796283 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:45.796283 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:45.796422 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:45.796465 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:45.796676 (+ 211us) server_negotiation.cc:299] Negotiation successful
0923 23:14:45.796759 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":243}
I20260923 23:14:45.797016 32546 heartbeater.cc:511] Master 127.24.227.254:41951 requested a full tablet report, sending...
I20260923 23:14:45.797258 32549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:45.784894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.253:35161 (local address 127.24.227.193:51243)
0923 23:14:45.785347 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:45.785355 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:45.786130 (+ 775us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:45.786375 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:45.786378 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:45.786806 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:14:45.787320 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:45.787323 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:45.788138 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:45.788139 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:45.789013 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:45.789016 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:45.789541 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:45.789542 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:45.789624 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:45.790295 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:14:45.790301 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:14:45.792379 (+ 2078us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:14:45.795663 (+ 3284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:45.795664 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:45.795664 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:45.795945 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:45.796149 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:45.796150 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:45.796150 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:45.796208 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:45.796515 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:14:45.796516 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:14:45.796570 (+ 54us) client_negotiation.cc:769] Sending connection context
0923 23:14:45.797006 (+ 436us) client_negotiation.cc:240] Negotiation successful
0923 23:14:45.797112 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":422,"mutex_wait_us":34,"thread_start_us":90,"threads_started":1}
I20260923 23:14:45.797590 31572 ts_manager.cc:194] Re-registered known tserver with Master: 85902a6195e544b7a8e8794e0b7c7ca1 (127.24.227.193:42775)
I20260923 23:14:45.797590 32547 heartbeater.cc:347] Connected to a master server at 127.24.227.253:35161
I20260923 23:14:45.797644 32547 heartbeater.cc:464] Registering TS with master...
I20260923 23:14:45.797719 32547 heartbeater.cc:511] Master 127.24.227.253:35161 requested a full tablet report, sending...
I20260923 23:14:45.798038 31572 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.227.193@KRBTEST.COM'} at 127.24.227.193:42519
I20260923 23:14:45.798051 32177 ts_manager.cc:194] Registered new tserver with Master: 85902a6195e544b7a8e8794e0b7c7ca1 (127.24.227.193:42775)
I20260923 23:14:45.798943 32415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:45.784976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.193:56261 (local address 127.24.227.252:46859)
0923 23:14:45.785236 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:45.785238 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:45.786342 (+ 1104us) server_negotiation.cc:407] Connection header received
0923 23:14:45.786374 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:45.786375 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:45.786407 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:45.786464 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:45.787444 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:45.788085 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:45.789683 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:45.790100 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:45.790153 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:45.793029 (+ 2876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:45.793030 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:45.793031 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:45.793038 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:45.795539 (+ 2501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:45.796268 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:45.796269 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:45.796270 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:45.796333 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:45.797286 (+ 953us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:45.797287 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:45.797287 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:45.797430 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:45.797474 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:45.798797 (+ 1323us) server_negotiation.cc:299] Negotiation successful
0923 23:14:45.798869 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":237}
I20260923 23:14:45.799115 32550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:45.784894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.252:46859 (local address 127.24.227.193:56261)
0923 23:14:45.786268 (+ 1374us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:45.786278 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:45.786326 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:45.786521 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:45.786522 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:45.786805 (+ 283us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:14:45.787344 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:45.787346 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:45.788173 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:45.788174 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:45.789589 (+ 1415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:45.789591 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:45.790233 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:45.790233 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:45.790300 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:45.790654 (+ 354us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:14:45.790659 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:14:45.792962 (+ 2303us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:14:45.795982 (+ 3020us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:45.795983 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:45.795983 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:45.796170 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:45.796671 (+ 501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:45.796672 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:45.796672 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:45.797199 (+ 527us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:45.798652 (+ 1453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:14:45.798653 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:14:45.798677 (+ 24us) client_negotiation.cc:769] Sending connection context
0923 23:14:45.798746 (+ 69us) client_negotiation.cc:240] Negotiation successful
0923 23:14:45.798852 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1349,"mutex_wait_us":29,"thread_start_us":86,"threads_started":1}
I20260923 23:14:45.799512 32545 heartbeater.cc:347] Connected to a master server at 127.24.227.252:46859
I20260923 23:14:45.799579 32545 heartbeater.cc:464] Registering TS with master...
I20260923 23:14:45.799645 32545 heartbeater.cc:511] Master 127.24.227.252:46859 requested a full tablet report, sending...
I20260923 23:14:45.800029 32364 ts_manager.cc:194] Registered new tserver with Master: 85902a6195e544b7a8e8794e0b7c7ca1 (127.24.227.193:42775)
W20260923 23:14:45.874260 32552 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:14:45.874478 32552 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:14:45.874572 32552 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:14:45.875736 32552 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:14:45.875771 32552 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:14:45.875815 32552 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.227.194
I20260923 23:14:45.877146 32552 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:42907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/krb5kdc/oryx_127.24.227.194.keytab
--principal=oryx/127.24.227.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.227.194:33365
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.227.194
--webserver_port=45683
--tserver_master_addrs=127.24.227.254:41951,127.24.227.253:35161,127.24.227.252:46859
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32552
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.227.194
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:14:45.877288 32552 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:14:45.877485 32552 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:14:45.881870 32560 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:45.881928 32558 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:45.881933 32557 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:14:45.882133 32552 server_base.cc:1034] running on GCE node
I20260923 23:14:45.882295 32552 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:14:45.882541 32552 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:14:45.883708 32552 hybrid_clock.cc:648] HybridClock initialized: now 1790205285883689 us; error 35 us; skew 500 ppm
Sep 23 23:14:45 dist-test-slave-754n krb5kdc[31528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205285, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.227.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:14:45.886023 32552 init.cc:377] Logged in from keytab as oryx/127.24.227.194@KRBTEST.COM (short username oryx)
I20260923 23:14:45.886335 32552 webserver.cc:460] Webserver started at http://127.24.227.194:45683/ using document root <none> and password file <none>
I20260923 23:14:45.886482 32552 fs_manager.cc:359] Metadata directory not provided
I20260923 23:14:45.886525 32552 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:14:45.888025 32552 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 23:14:45.888820 32567 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:45.889027 32552 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260923 23:14:45.889110 32552 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/ts-1/wal
uuid: "99b029603b8e4c9fbafedaafc91ea951"
format_stamp: "Formatted at 2026-09-23 23:14:44 on dist-test-slave-754n"
server_key: "b54c31f66bc175b2320a23c8263b7533"
server_key_iv: "3c49046c2ae5e8e0f7792fed3b6447ef"
server_key_version: "encryptionkey@0"
I20260923 23:14:45.889398 32552 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:14:45.895089 32552 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:14:45.896698 32552 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:14:45.896845 32552 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:14:45.897060 32552 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 23:14:45.897322 32552 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 23:14:45.897372 32552 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:45.897409 32552 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 23:14:45.897436 32552 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:45.902910 32552 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.194:33365
I20260923 23:14:45.902978 32680 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.194:33365 every 8 connection(s)
I20260923 23:14:45.903329 32552 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/ts-1/data/info.pb
Sep 23 23:14:45 dist-test-slave-754n krb5kdc[31528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205285, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.227.194@KRBTEST.COM for oryx/127.24.227.252@KRBTEST.COM
Sep 23 23:14:45 dist-test-slave-754n krb5kdc[31528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205285, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.227.194@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM
Sep 23 23:14:45 dist-test-slave-754n krb5kdc[31528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205285, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.227.194@KRBTEST.COM for oryx/127.24.227.253@KRBTEST.COM
I20260923 23:14:45.911355 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 32552
I20260923 23:14:45.912302 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 31903
I20260923 23:14:45.913120 32415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:45.904274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.194:52993 (local address 127.24.227.252:46859)
0923 23:14:45.904315 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:45.904317 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:45.904328 (+ 11us) server_negotiation.cc:407] Connection header received
0923 23:14:45.905476 (+ 1148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:45.905478 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:45.905487 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:45.905526 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:45.906284 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:45.907001 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:45.908028 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:45.908221 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:45.908276 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:45.910131 (+ 1855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:45.910133 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:45.910133 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:45.910140 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:45.911534 (+ 1394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:45.912055 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:45.912057 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:45.912058 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:45.912122 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:45.912390 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:45.912392 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:45.912392 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:45.912532 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:45.912594 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:45.912915 (+ 321us) server_negotiation.cc:299] Negotiation successful
0923 23:14:45.912990 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260923 23:14:45.913600 32685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:45.903995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.252:46859 (local address 127.24.227.194:52993)
0923 23:14:45.904197 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:45.904236 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:45.905242 (+ 1006us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:45.905565 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:45.905566 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:45.905990 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:14:45.906184 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:45.906190 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:45.907082 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:45.907082 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:45.907949 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:45.907950 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:45.908346 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:45.908347 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:45.908437 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:45.908736 (+ 299us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:14:45.908741 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:14:45.910049 (+ 1308us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:14:45.911643 (+ 1594us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:45.911656 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:45.911658 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:45.911994 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:45.912178 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:45.912178 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:45.912179 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:45.912305 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:45.912625 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:14:45.912631 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:14:45.912773 (+ 142us) client_negotiation.cc:769] Sending connection context
0923 23:14:45.913194 (+ 421us) client_negotiation.cc:240] Negotiation successful
0923 23:14:45.913355 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"mutex_wait_us":40,"thread_start_us":117,"threads_started":1}
I20260923 23:14:45.914078 31613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:45.904346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.194:40235 (local address 127.24.227.254:41951)
0923 23:14:45.904427 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:45.904429 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:45.904635 (+ 206us) server_negotiation.cc:407] Connection header received
0923 23:14:45.905586 (+ 951us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:45.905587 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:45.905596 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:45.905677 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:45.906273 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:45.906724 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:45.907677 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:45.907853 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:45.907953 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:45.911179 (+ 3226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:45.911180 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:45.911181 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:45.911187 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:45.912848 (+ 1661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:45.913253 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:45.913254 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:45.913255 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:45.913311 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:45.913475 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:45.913476 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:45.913476 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:45.913614 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:45.913658 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:45.913894 (+ 236us) server_negotiation.cc:299] Negotiation successful
0923 23:14:45.913974 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 23:14:45.914315 32681 heartbeater.cc:347] Connected to a master server at 127.24.227.252:46859
I20260923 23:14:45.914418 32681 heartbeater.cc:464] Registering TS with master...
I20260923 23:14:45.914431 32687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:45.903995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41951 (local address 127.24.227.194:40235)
0923 23:14:45.904575 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:45.904586 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:45.905257 (+ 671us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:45.905694 (+ 437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:45.905695 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:45.905998 (+ 303us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:14:45.906184 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:45.906189 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:45.906842 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:45.906843 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:45.907579 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:45.907581 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:45.907961 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:45.907962 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:45.908093 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:45.908624 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:14:45.908633 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:14:45.911085 (+ 2452us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:14:45.912947 (+ 1862us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:45.912948 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:45.912949 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:45.913167 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:45.913372 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:45.913373 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:45.913373 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:45.913420 (+ 47us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:45.913724 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:14:45.913725 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:14:45.913783 (+ 58us) client_negotiation.cc:769] Sending connection context
0923 23:14:45.914199 (+ 416us) client_negotiation.cc:240] Negotiation successful
0923 23:14:45.914288 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":557,"mutex_wait_us":28,"thread_start_us":120,"threads_started":1}
I20260923 23:14:45.914626 32681 heartbeater.cc:511] Master 127.24.227.252:46859 requested a full tablet report, sending...
I20260923 23:14:45.914899 32682 heartbeater.cc:347] Connected to a master server at 127.24.227.254:41951
I20260923 23:14:45.914964 32682 heartbeater.cc:464] Registering TS with master...
I20260923 23:14:45.915035 32682 heartbeater.cc:511] Master 127.24.227.254:41951 requested a full tablet report, sending...
I20260923 23:14:45.915132 32364 ts_manager.cc:194] Registered new tserver with Master: 99b029603b8e4c9fbafedaafc91ea951 (127.24.227.194:33365)
I20260923 23:14:45.915261 31572 ts_manager.cc:194] Re-registered known tserver with Master: 99b029603b8e4c9fbafedaafc91ea951 (127.24.227.194:33365)
I20260923 23:14:45.915740 31572 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.227.194@KRBTEST.COM'} at 127.24.227.194:40235
I20260923 23:14:45.916013 32686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:45.903995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.253:35161 (local address 127.24.227.194:47909)
0923 23:14:45.904274 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:45.904282 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:45.905242 (+ 960us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:45.905478 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:45.905481 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:45.905952 (+ 471us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:14:45.906184 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:45.906190 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:45.907082 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:45.907082 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:45.907920 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:45.907923 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:45.908405 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:45.908406 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:45.908501 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:45.908912 (+ 411us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:14:45.908916 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:14:45.911126 (+ 2210us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:14:45.913489 (+ 2363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:45.913490 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:45.913490 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:45.914109 (+ 619us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:45.915303 (+ 1194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:45.915303 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:45.915304 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:45.915372 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:45.915672 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:14:45.915672 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:14:45.915722 (+ 50us) client_negotiation.cc:769] Sending connection context
0923 23:14:45.915798 (+ 76us) client_negotiation.cc:240] Negotiation successful
0923 23:14:45.915924 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"mutex_wait_us":31,"thread_start_us":134,"threads_started":1}
I20260923 23:14:45.916113 32228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:45.904293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.194:47909 (local address 127.24.227.253:35161)
0923 23:14:45.904397 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:45.904399 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:45.904432 (+ 33us) server_negotiation.cc:407] Connection header received
0923 23:14:45.905351 (+ 919us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:45.905352 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:45.905367 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:45.905425 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:45.906284 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:45.906988 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:45.908022 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:45.908302 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:45.908404 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:45.911200 (+ 2796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:45.911201 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:45.911202 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:45.911208 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:45.912896 (+ 1688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:45.914890 (+ 1994us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:45.914891 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:45.914891 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:45.914956 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:45.915439 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:45.915440 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:45.915441 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:45.915599 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:45.915671 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:45.915914 (+ 243us) server_negotiation.cc:299] Negotiation successful
0923 23:14:45.915994 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260923 23:14:45.918270 32683 heartbeater.cc:347] Connected to a master server at 127.24.227.253:35161
I20260923 23:14:45.918322 32683 heartbeater.cc:464] Registering TS with master...
I20260923 23:14:45.918406 32683 heartbeater.cc:511] Master 127.24.227.253:35161 requested a full tablet report, sending...
I20260923 23:14:45.918718 32177 ts_manager.cc:194] Registered new tserver with Master: 99b029603b8e4c9fbafedaafc91ea951 (127.24.227.194:33365)
I20260923 23:14:45.921465 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.227.195:43865
--local_ip_for_outbound_sockets=127.24.227.195
--tserver_master_addrs=127.24.227.254:41951
--webserver_port=46125
--webserver_interface=127.24.227.195
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/krb5kdc/oryx_127.24.227.195.keytab
--principal=oryx/127.24.227.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:42907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.24.227.254:41951,127.24.227.253:35161
--tserver_master_addrs=127.24.227.254:41951,127.24.227.253:35161,127.24.227.252:46859 with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:14:45.998109 32688 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:14:45.998289 32688 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:14:45.998312 32688 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:14:45.999365 32688 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:14:45.999398 32688 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:14:45.999423 32688 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.227.195
I20260923 23:14:46.000823 32688 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:42907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/krb5kdc/oryx_127.24.227.195.keytab
--principal=oryx/127.24.227.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.227.195:43865
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.227.195
--webserver_port=46125
--tserver_master_addrs=127.24.227.254:41951,127.24.227.253:35161,127.24.227.252:46859
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32688
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.227.195
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:14:46.000986 32688 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:14:46.001138 32688 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:14:46.005795 32693 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:46.005826 32696 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:46.005805 32694 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:14:46.006107 32688 server_base.cc:1034] running on GCE node
I20260923 23:14:46.006309 32688 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:14:46.006529 32688 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:14:46.007683 32688 hybrid_clock.cc:648] HybridClock initialized: now 1790205286007665 us; error 45 us; skew 500 ppm
Sep 23 23:14:46 dist-test-slave-754n krb5kdc[31528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205286, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.227.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:14:46.015589 32688 init.cc:377] Logged in from keytab as oryx/127.24.227.195@KRBTEST.COM (short username oryx)
I20260923 23:14:46.016067 32688 webserver.cc:460] Webserver started at http://127.24.227.195:46125/ using document root <none> and password file <none>
I20260923 23:14:46.016299 32688 fs_manager.cc:359] Metadata directory not provided
I20260923 23:14:46.016374 32688 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:14:46.017680 32688 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 23:14:46.018260 32703 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:46.018417 32688 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:46.018504 32688 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/ts-2/wal
uuid: "6eb81312874d421c9884f64eff92eae6"
format_stamp: "Formatted at 2026-09-23 23:14:45 on dist-test-slave-754n"
server_key: "4f4e8217e5135576fe5e10d4ccbe4579"
server_key_iv: "3f24b987af94d1e044e89b73e7e49be5"
server_key_version: "encryptionkey@0"
I20260923 23:14:46.018862 32688 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:14:46.032335 32688 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:14:46.033908 32415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:46.030989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41586 (local address 127.24.227.252:46859)
0923 23:14:46.031040 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:46.031042 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:46.031055 (+ 13us) server_negotiation.cc:407] Connection header received
0923 23:14:46.031139 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:46.031140 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:46.031162 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:46.031230 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:14:46.031342 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:46.032064 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:46.033037 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:46.033623 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:46.033697 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:46.033759 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:46.033785 (+ 26us) server_negotiation.cc:299] Negotiation successful
0923 23:14:46.033823 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260923 23:14:46.033926 32131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:46.030921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.252:46859 (local address 127.0.0.1:41586)
0923 23:14:46.031005 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:46.031016 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:46.031067 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:46.031238 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:46.031239 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:46.031241 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:14:46.031280 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:46.031282 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:46.032128 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:46.032129 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:46.032933 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:46.032935 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:46.033700 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:46.033701 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:46.033744 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:46.033747 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 23:14:46.033781 (+ 34us) client_negotiation.cc:240] Negotiation successful
0923 23:14:46.033811 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260923 23:14:46.034422 32374 raft_consensus.cc:1203] T 00000000000000000000000000000000 P f014784de6dd46128a093784ebbe68fb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260923 23:14:46.034626 32688 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:14:46.034605 32403 sys_catalog.cc:455] T 00000000000000000000000000000000 P f014784de6dd46128a093784ebbe68fb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ddae489e6844395a47c75343be04fa5. Latest consensus state: current_term: 1 leader_uuid: "2ddae489e6844395a47c75343be04fa5" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2ddae489e6844395a47c75343be04fa5" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41951 } } peers { permanent_uuid: "bd675df8e6274c688bb00c0f39e018c1" member_type: VOTER last_known_addr { host: "127.24.227.253" port: 35161 } attrs { promote: false } } peers { permanent_uuid: "f014784de6dd46128a093784ebbe68fb" member_type: NON_VOTER last_known_addr { host: "127.24.227.252" port: 46859 } attrs { promote: true } } }
I20260923 23:14:46.034734 32403 sys_catalog.cc:458] T 00000000000000000000000000000000 P f014784de6dd46128a093784ebbe68fb [sys.catalog]: This master's current role is: LEARNER
I20260923 23:14:46.034775 32688 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:14:46.034981 32688 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 23:14:46.035211 32688 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 23:14:46.035240 32688 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:46.035260 32688 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 23:14:46.035272 32688 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:46.040531 32688 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.195:43865
I20260923 23:14:46.040573 348 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.195:43865 every 8 connection(s)
I20260923 23:14:46.040877 32688 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/ts-2/data/info.pb
I20260923 23:14:46.045648 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 32688
Sep 23 23:14:46 dist-test-slave-754n krb5kdc[31528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205286, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.227.195@KRBTEST.COM for oryx/127.24.227.253@KRBTEST.COM
Sep 23 23:14:46 dist-test-slave-754n krb5kdc[31528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205286, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.227.195@KRBTEST.COM for oryx/127.24.227.252@KRBTEST.COM
Sep 23 23:14:46 dist-test-slave-754n krb5kdc[31528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205286, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.227.195@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM
I20260923 23:14:46.052145 31613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:46.041452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.195:60119 (local address 127.24.227.254:41951)
0923 23:14:46.041495 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:46.041498 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:46.041761 (+ 263us) server_negotiation.cc:407] Connection header received
0923 23:14:46.042815 (+ 1054us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:46.042816 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:46.042828 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:46.042857 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:46.043559 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:46.044316 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:46.045424 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:46.046053 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:46.046130 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:46.049142 (+ 3012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:46.049143 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:46.049144 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:46.049150 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:46.050661 (+ 1511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:46.051083 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:46.051084 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:46.051084 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:46.051156 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:46.051418 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:46.051418 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:46.051419 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:46.051537 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:46.051592 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:46.051969 (+ 377us) server_negotiation.cc:299] Negotiation successful
0923 23:14:46.052047 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260923 23:14:46.052377 355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:46.041758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.252:46859 (local address 127.24.227.195:50953)
0923 23:14:46.041872 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:46.041882 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:46.042722 (+ 840us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:46.042942 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:46.042943 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:46.043297 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:14:46.043459 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:46.043465 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:46.044413 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:46.044414 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:46.045366 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:46.045367 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:46.045875 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:46.045876 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:46.045946 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:46.046488 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:14:46.046494 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:14:46.048456 (+ 1962us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:14:46.051084 (+ 2628us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:46.051085 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:46.051086 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:46.051336 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:46.051546 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:46.051547 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:46.051547 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:46.051604 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:46.051889 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:14:46.051890 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:14:46.051930 (+ 40us) client_negotiation.cc:769] Sending connection context
0923 23:14:46.052000 (+ 70us) client_negotiation.cc:240] Negotiation successful
0923 23:14:46.052160 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"mutex_wait_us":35,"thread_start_us":73,"threads_started":1}
I20260923 23:14:46.052687 354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:46.041449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41951 (local address 127.24.227.195:60119)
0923 23:14:46.041634 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:46.041672 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:46.042725 (+ 1053us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:46.042942 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:46.042943 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:46.043335 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:14:46.043459 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:46.043465 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:46.044413 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:46.044414 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:46.045353 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:46.045356 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:46.046182 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:46.046183 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:46.046502 (+ 319us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:46.046885 (+ 383us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:14:46.046890 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:14:46.049051 (+ 2161us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:14:46.050738 (+ 1687us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:46.050744 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:46.050746 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:46.050990 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:46.051222 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:46.051223 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:46.051224 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:46.051347 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:46.051643 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:14:46.051648 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:14:46.051790 (+ 142us) client_negotiation.cc:769] Sending connection context
0923 23:14:46.052350 (+ 560us) client_negotiation.cc:240] Negotiation successful
0923 23:14:46.052585 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"mutex_wait_us":101,"thread_start_us":78,"threads_started":1}
I20260923 23:14:46.053236 32415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:46.041970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.195:50953 (local address 127.24.227.252:46859)
0923 23:14:46.042027 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:46.042028 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:46.042036 (+ 8us) server_negotiation.cc:407] Connection header received
0923 23:14:46.042814 (+ 778us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:46.042815 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:46.042826 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:46.042857 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:46.043558 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:46.044316 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:46.045458 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:46.045756 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:46.045822 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:46.048633 (+ 2811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:46.048634 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:46.048635 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:46.048641 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:46.050992 (+ 2351us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:46.051417 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:46.051419 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:46.051419 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:46.051475 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:46.051667 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:46.051668 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:46.051668 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:46.051784 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:46.051839 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:46.053036 (+ 1197us) server_negotiation.cc:299] Negotiation successful
0923 23:14:46.053126 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260923 23:14:46.053551 349 heartbeater.cc:347] Connected to a master server at 127.24.227.252:46859
I20260923 23:14:46.053668 349 heartbeater.cc:464] Registering TS with master...
I20260923 23:14:46.053686 350 heartbeater.cc:347] Connected to a master server at 127.24.227.254:41951
I20260923 23:14:46.053714 350 heartbeater.cc:464] Registering TS with master...
I20260923 23:14:46.053839 350 heartbeater.cc:511] Master 127.24.227.254:41951 requested a full tablet report, sending...
I20260923 23:14:46.053839 349 heartbeater.cc:511] Master 127.24.227.252:46859 requested a full tablet report, sending...
I20260923 23:14:46.054257 32364 ts_manager.cc:194] Registered new tserver with Master: 6eb81312874d421c9884f64eff92eae6 (127.24.227.195:43865)
I20260923 23:14:46.054582 31572 ts_manager.cc:194] Re-registered known tserver with Master: 6eb81312874d421c9884f64eff92eae6 (127.24.227.195:43865)
I20260923 23:14:46.054914 31572 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.227.195@KRBTEST.COM'} at 127.24.227.195:60119
I20260923 23:14:46.056036 32228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:46.041477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.195:41899 (local address 127.24.227.253:35161)
0923 23:14:46.041540 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:46.041542 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:46.041782 (+ 240us) server_negotiation.cc:407] Connection header received
0923 23:14:46.042803 (+ 1021us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:46.042804 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:46.042816 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:46.042866 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:46.043551 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:46.044088 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:46.045066 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:46.045269 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:46.045349 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:46.052971 (+ 7622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:46.052973 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:46.052973 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:46.052980 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:46.054820 (+ 1840us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:46.055259 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:46.055260 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:46.055260 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:46.055313 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:46.055512 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:46.055513 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:46.055514 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:46.055619 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:46.055690 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:46.055853 (+ 163us) server_negotiation.cc:299] Negotiation successful
0923 23:14:46.055943 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260923 23:14:46.056627 353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:46.041448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.253:35161 (local address 127.24.227.195:41899)
0923 23:14:46.041633 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:46.041670 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:46.042701 (+ 1031us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:46.042937 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:46.042941 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:46.043291 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:14:46.043459 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:46.043462 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:46.044199 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:46.044200 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:46.044965 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:46.044967 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:46.045360 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:46.045361 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:46.045523 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:46.046009 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:14:46.046018 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:14:46.049962 (+ 3944us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:14:46.054999 (+ 5037us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:46.055000 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:46.055001 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:46.055201 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:46.055386 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:46.055386 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:46.055387 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:46.055440 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:46.055683 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:14:46.055683 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:14:46.055734 (+ 51us) client_negotiation.cc:769] Sending connection context
0923 23:14:46.055831 (+ 97us) client_negotiation.cc:240] Negotiation successful
0923 23:14:46.055944 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":84,"threads_started":1}
I20260923 23:14:46.057090 351 heartbeater.cc:347] Connected to a master server at 127.24.227.253:35161
I20260923 23:14:46.057133 351 heartbeater.cc:464] Registering TS with master...
I20260923 23:14:46.057195 351 heartbeater.cc:511] Master 127.24.227.253:35161 requested a full tablet report, sending...
I20260923 23:14:46.057490 32177 ts_manager.cc:194] Registered new tserver with Master: 6eb81312874d421c9884f64eff92eae6 (127.24.227.195:43865)
I20260923 23:14:46.057945 361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:46.047043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35700 (local address 127.24.227.254:41951)
0923 23:14:46.047211 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:46.047212 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:46.047724 (+ 512us) server_negotiation.cc:407] Connection header received
0923 23:14:46.047773 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:46.047774 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:46.047788 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:46.048717 (+ 929us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:46.048749 (+ 32us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:46.050107 (+ 1358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:46.051357 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:46.051928 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:46.053727 (+ 1799us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:46.053773 (+ 46us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:46.053774 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:46.053775 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:46.053779 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:46.056291 (+ 2512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:46.056668 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:46.056669 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:46.056669 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:46.056721 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:46.056938 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:46.056939 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:46.056939 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:46.057043 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:46.057715 (+ 672us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:46.057757 (+ 42us) server_negotiation.cc:299] Negotiation successful
0923 23:14:46.057838 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":121,"threads_started":1}
I20260923 23:14:46.058176 363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:46.047535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50658 (local address 127.24.227.253:35161)
0923 23:14:46.047814 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:46.047815 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:46.048010 (+ 195us) server_negotiation.cc:407] Connection header received
0923 23:14:46.048087 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:46.048087 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:46.048098 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:46.048157 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:46.049127 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:46.049839 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:46.050804 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:46.051123 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:46.051218 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:46.054102 (+ 2884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:46.054103 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:46.054103 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:46.054110 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:46.056815 (+ 2705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:46.057228 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:46.057228 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:46.057229 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:46.057284 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:46.057578 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:46.057579 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:46.057579 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:46.057699 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:46.057775 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:46.057878 (+ 103us) server_negotiation.cc:299] Negotiation successful
0923 23:14:46.057950 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261,"thread_start_us":59,"threads_started":1}
I20260923 23:14:46.061395 364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:46.047753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41594 (local address 127.24.227.252:46859)
0923 23:14:46.047867 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:46.047869 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:46.049224 (+ 1355us) server_negotiation.cc:407] Connection header received
0923 23:14:46.049303 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:46.049304 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:46.049313 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:46.049347 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:46.051586 (+ 2239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:46.052336 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:46.056823 (+ 4487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:46.057121 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:46.057157 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:46.058830 (+ 1673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:46.058831 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:46.058831 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:46.058838 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:46.060348 (+ 1510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:46.060740 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:46.060741 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:46.060742 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:46.060826 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:46.060989 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:46.060990 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:46.060990 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:46.061098 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:46.061195 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:46.061250 (+ 55us) server_negotiation.cc:299] Negotiation successful
0923 23:14:46.061318 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":53,"threads_started":1}
I20260923 23:14:46.059401 31572 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35700:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 23:14:46.061772 31572 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 23:14:46.062340 32132 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ddae489e6844395a47c75343be04fa5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2ddae489e6844395a47c75343be04fa5" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2ddae489e6844395a47c75343be04fa5" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41951 } } peers { permanent_uuid: "bd675df8e6274c688bb00c0f39e018c1" member_type: VOTER last_known_addr { host: "127.24.227.253" port: 35161 } attrs { promote: false } } peers { permanent_uuid: "f014784de6dd46128a093784ebbe68fb" member_type: NON_VOTER last_known_addr { host: "127.24.227.252" port: 46859 } attrs { promote: true } } }
I20260923 23:14:46.062433 32132 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ddae489e6844395a47c75343be04fa5 [sys.catalog]: This master's current role is: LEADER
I20260923 23:14:46.063808 32132 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 2ddae489e6844395a47c75343be04fa5: attempting to promote NON_VOTER f014784de6dd46128a093784ebbe68fb to VOTER
I20260923 23:14:46.063975 32132 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2ddae489e6844395a47c75343be04fa5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "2ddae489e6844395a47c75343be04fa5" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41951 } } peers { permanent_uuid: "bd675df8e6274c688bb00c0f39e018c1" member_type: VOTER last_known_addr { host: "127.24.227.253" port: 35161 } attrs { promote: false } } peers { permanent_uuid: "f014784de6dd46128a093784ebbe68fb" member_type: VOTER last_known_addr { host: "127.24.227.252" port: 46859 } attrs { promote: false } }
I20260923 23:14:46.064327 32187 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bd675df8e6274c688bb00c0f39e018c1 [term 1 FOLLOWER]: Refusing update from remote peer 2ddae489e6844395a47c75343be04fa5: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20260923 23:14:46.064621 32374 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f014784de6dd46128a093784ebbe68fb [term 1 LEARNER]: Refusing update from remote peer 2ddae489e6844395a47c75343be04fa5: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20260923 23:14:46.064538 376 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2ddae489e6844395a47c75343be04fa5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd675df8e6274c688bb00c0f39e018c1" member_type: VOTER last_known_addr { host: "127.24.227.253" port: 35161 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260923 23:14:46.064817 367 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2ddae489e6844395a47c75343be04fa5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f014784de6dd46128a093784ebbe68fb" member_type: VOTER last_known_addr { host: "127.24.227.252" port: 46859 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260923 23:14:46.065523 376 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 2ddae489e6844395a47c75343be04fa5 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, f014784de6dd46128a093784ebbe68fb (127.24.227.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "2ddae489e6844395a47c75343be04fa5" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41951 } } peers { permanent_uuid: "bd675df8e6274c688bb00c0f39e018c1" member_type: VOTER last_known_addr { host: "127.24.227.253" port: 35161 } attrs { promote: false } } peers { permanent_uuid: "f014784de6dd46128a093784ebbe68fb" member_type: VOTER last_known_addr { host: "127.24.227.252" port: 46859 } attrs { promote: false } } }
I20260923 23:14:46.065650 32187 raft_consensus.cc:2932] T 00000000000000000000000000000000 P bd675df8e6274c688bb00c0f39e018c1 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, f014784de6dd46128a093784ebbe68fb (127.24.227.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "2ddae489e6844395a47c75343be04fa5" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41951 } } peers { permanent_uuid: "bd675df8e6274c688bb00c0f39e018c1" member_type: VOTER last_known_addr { host: "127.24.227.253" port: 35161 } attrs { promote: false } } peers { permanent_uuid: "f014784de6dd46128a093784ebbe68fb" member_type: VOTER last_known_addr { host: "127.24.227.252" port: 46859 } attrs { promote: false } } }
I20260923 23:14:46.065663 32374 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f014784de6dd46128a093784ebbe68fb [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, f014784de6dd46128a093784ebbe68fb (127.24.227.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "2ddae489e6844395a47c75343be04fa5" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41951 } } peers { permanent_uuid: "bd675df8e6274c688bb00c0f39e018c1" member_type: VOTER last_known_addr { host: "127.24.227.253" port: 35161 } attrs { promote: false } } peers { permanent_uuid: "f014784de6dd46128a093784ebbe68fb" member_type: VOTER last_known_addr { host: "127.24.227.252" port: 46859 } attrs { promote: false } } }
I20260923 23:14:46.066576 32403 sys_catalog.cc:455] T 00000000000000000000000000000000 P f014784de6dd46128a093784ebbe68fb [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2ddae489e6844395a47c75343be04fa5" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "2ddae489e6844395a47c75343be04fa5" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41951 } } peers { permanent_uuid: "bd675df8e6274c688bb00c0f39e018c1" member_type: VOTER last_known_addr { host: "127.24.227.253" port: 35161 } attrs { promote: false } } peers { permanent_uuid: "f014784de6dd46128a093784ebbe68fb" member_type: VOTER last_known_addr { host: "127.24.227.252" port: 46859 } attrs { promote: false } } }
I20260923 23:14:46.066593 378 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd675df8e6274c688bb00c0f39e018c1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2ddae489e6844395a47c75343be04fa5" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "2ddae489e6844395a47c75343be04fa5" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41951 } } peers { permanent_uuid: "bd675df8e6274c688bb00c0f39e018c1" member_type: VOTER last_known_addr { host: "127.24.227.253" port: 35161 } attrs { promote: false } } peers { permanent_uuid: "f014784de6dd46128a093784ebbe68fb" member_type: VOTER last_known_addr { host: "127.24.227.252" port: 46859 } attrs { promote: false } } }
I20260923 23:14:46.066668 32403 sys_catalog.cc:458] T 00000000000000000000000000000000 P f014784de6dd46128a093784ebbe68fb [sys.catalog]: This master's current role is: FOLLOWER
I20260923 23:14:46.066685 378 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd675df8e6274c688bb00c0f39e018c1 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 23:14:46.066668 369 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ddae489e6844395a47c75343be04fa5 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "2ddae489e6844395a47c75343be04fa5" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "2ddae489e6844395a47c75343be04fa5" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41951 } } peers { permanent_uuid: "bd675df8e6274c688bb00c0f39e018c1" member_type: VOTER last_known_addr { host: "127.24.227.253" port: 35161 } attrs { promote: false } } peers { permanent_uuid: "f014784de6dd46128a093784ebbe68fb" member_type: VOTER last_known_addr { host: "127.24.227.252" port: 46859 } attrs { promote: false } } }
I20260923 23:14:46.066747 369 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ddae489e6844395a47c75343be04fa5 [sys.catalog]: This master's current role is: LEADER
I20260923 23:14:46.066998 32132 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ddae489e6844395a47c75343be04fa5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2ddae489e6844395a47c75343be04fa5" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "2ddae489e6844395a47c75343be04fa5" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41951 } } peers { permanent_uuid: "bd675df8e6274c688bb00c0f39e018c1" member_type: VOTER last_known_addr { host: "127.24.227.253" port: 35161 } attrs { promote: false } } peers { permanent_uuid: "f014784de6dd46128a093784ebbe68fb" member_type: VOTER last_known_addr { host: "127.24.227.252" port: 46859 } attrs { promote: false } } }
I20260923 23:14:46.066977 369 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ddae489e6844395a47c75343be04fa5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2ddae489e6844395a47c75343be04fa5" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "2ddae489e6844395a47c75343be04fa5" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41951 } } peers { permanent_uuid: "bd675df8e6274c688bb00c0f39e018c1" member_type: VOTER last_known_addr { host: "127.24.227.253" port: 35161 } attrs { promote: false } } peers { permanent_uuid: "f014784de6dd46128a093784ebbe68fb" member_type: VOTER last_known_addr { host: "127.24.227.252" port: 46859 } attrs { promote: false } } }
I20260923 23:14:46.067057 32132 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ddae489e6844395a47c75343be04fa5 [sys.catalog]: This master's current role is: LEADER
I20260923 23:14:46.067071 369 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ddae489e6844395a47c75343be04fa5 [sys.catalog]: This master's current role is: LEADER
I20260923 23:14:46.071049 32131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:46.067334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.193:42775 (local address 127.0.0.1:46032)
0923 23:14:46.067385 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:46.067395 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:46.067439 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:46.067772 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:46.067773 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:46.067774 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:14:46.067806 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:46.067807 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:46.069065 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:46.069066 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:46.070106 (+ 1040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:46.070108 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:46.070824 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:46.070825 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:46.070884 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:46.070887 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 23:14:46.070937 (+ 50us) client_negotiation.cc:240] Negotiation successful
0923 23:14:46.070974 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34,"mutex_wait_us":39}
I20260923 23:14:46.071050 379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:46.067428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.194:33365 (local address 127.0.0.1:35268)
0923 23:14:46.067556 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:46.067565 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:46.067616 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:46.068197 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:46.068198 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:46.068199 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:14:46.068225 (+ 26us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:46.068227 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:46.069281 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:46.069282 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:46.070001 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:46.070003 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:46.070862 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:46.070862 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:46.070910 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:46.070913 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 23:14:46.070962 (+ 49us) client_negotiation.cc:240] Negotiation successful
0923 23:14:46.070985 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":89,"threads_started":1}
I20260923 23:14:46.071070 380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:46.067464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46032 (local address 127.24.227.193:42775)
0923 23:14:46.067603 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:46.067607 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:46.067617 (+ 10us) server_negotiation.cc:407] Connection header received
0923 23:14:46.067652 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:46.067654 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:46.067669 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:46.067912 (+ 243us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:14:46.067974 (+ 62us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:46.068994 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:46.070172 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:46.070762 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:46.070847 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:46.070899 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:46.070968 (+ 69us) server_negotiation.cc:299] Negotiation successful
0923 23:14:46.071002 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":72,"threads_started":1}
I20260923 23:14:46.071309 383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:46.067791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35268 (local address 127.24.227.194:33365)
0923 23:14:46.068021 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:46.068025 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:46.068034 (+ 9us) server_negotiation.cc:407] Connection header received
0923 23:14:46.068068 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:46.068070 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:46.068096 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:46.068205 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:14:46.068302 (+ 97us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:46.069214 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:46.070097 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:46.070753 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:46.070868 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:46.070912 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:46.071180 (+ 268us) server_negotiation.cc:299] Negotiation successful
0923 23:14:46.071233 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":122,"threads_started":1}
I20260923 23:14:46.071707 32479 tablet_service.cc:1467] Processing CreateTablet for tablet c88663338a774085b8e9fccba5b86750 (DEFAULT_TABLE table=test-table [id=ab8e1379b1a44886a3238a168520c134]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 23:14:46.071849 32615 tablet_service.cc:1467] Processing CreateTablet for tablet c88663338a774085b8e9fccba5b86750 (DEFAULT_TABLE table=test-table [id=ab8e1379b1a44886a3238a168520c134]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 23:14:46.072054 32479 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c88663338a774085b8e9fccba5b86750. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:14:46.072075 381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:46.067606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.195:43865 (local address 127.0.0.1:48976)
0923 23:14:46.067695 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:46.067704 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:46.067737 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:46.068015 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:46.068016 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:46.068017 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:14:46.068056 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:46.068057 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:46.069014 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:46.069015 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:46.070106 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:46.070115 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:46.071828 (+ 1713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:46.071829 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:46.071905 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:46.071909 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 23:14:46.071969 (+ 60us) client_negotiation.cc:240] Negotiation successful
0923 23:14:46.072005 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":76,"mutex_wait_us":29,"thread_start_us":56,"threads_started":1}
I20260923 23:14:46.072073 382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:46.067668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48976 (local address 127.24.227.195:43865)
0923 23:14:46.067832 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:46.067837 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:46.067846 (+ 9us) server_negotiation.cc:407] Connection header received
0923 23:14:46.067896 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:46.067898 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:46.067913 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:46.068011 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:14:46.068108 (+ 97us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:46.068942 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:46.070941 (+ 1999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:46.071744 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:46.071825 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:46.071891 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:46.071981 (+ 90us) server_negotiation.cc:299] Negotiation successful
0923 23:14:46.072011 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":65,"threads_started":1}
I20260923 23:14:46.072211 32615 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c88663338a774085b8e9fccba5b86750. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:14:46.074657 384 tablet_bootstrap.cc:492] T c88663338a774085b8e9fccba5b86750 P 85902a6195e544b7a8e8794e0b7c7ca1: Bootstrap starting.
I20260923 23:14:46.075006 385 tablet_bootstrap.cc:492] T c88663338a774085b8e9fccba5b86750 P 99b029603b8e4c9fbafedaafc91ea951: Bootstrap starting.
I20260923 23:14:46.075270 384 tablet_bootstrap.cc:654] T c88663338a774085b8e9fccba5b86750 P 85902a6195e544b7a8e8794e0b7c7ca1: Neither blocks nor log segments found. Creating new log.
I20260923 23:14:46.075486 384 log.cc:824] T c88663338a774085b8e9fccba5b86750 P 85902a6195e544b7a8e8794e0b7c7ca1: Log is configured to *not* fsync() on all Append() calls
I20260923 23:14:46.075585 385 tablet_bootstrap.cc:654] T c88663338a774085b8e9fccba5b86750 P 99b029603b8e4c9fbafedaafc91ea951: Neither blocks nor log segments found. Creating new log.
I20260923 23:14:46.075520 32751 tablet_service.cc:1467] Processing CreateTablet for tablet c88663338a774085b8e9fccba5b86750 (DEFAULT_TABLE table=test-table [id=ab8e1379b1a44886a3238a168520c134]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 23:14:46.075826 32751 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c88663338a774085b8e9fccba5b86750. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:14:46.075870 385 log.cc:824] T c88663338a774085b8e9fccba5b86750 P 99b029603b8e4c9fbafedaafc91ea951: Log is configured to *not* fsync() on all Append() calls
I20260923 23:14:46.076244 384 tablet_bootstrap.cc:492] T c88663338a774085b8e9fccba5b86750 P 85902a6195e544b7a8e8794e0b7c7ca1: No bootstrap required, opened a new log
I20260923 23:14:46.076315 384 ts_tablet_manager.cc:1397] T c88663338a774085b8e9fccba5b86750 P 85902a6195e544b7a8e8794e0b7c7ca1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 23:14:46.076848 385 tablet_bootstrap.cc:492] T c88663338a774085b8e9fccba5b86750 P 99b029603b8e4c9fbafedaafc91ea951: No bootstrap required, opened a new log
I20260923 23:14:46.076908 385 ts_tablet_manager.cc:1397] T c88663338a774085b8e9fccba5b86750 P 99b029603b8e4c9fbafedaafc91ea951: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260923 23:14:46.078147 384 raft_consensus.cc:348] T c88663338a774085b8e9fccba5b86750 P 85902a6195e544b7a8e8794e0b7c7ca1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85902a6195e544b7a8e8794e0b7c7ca1" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 42775 } } peers { permanent_uuid: "99b029603b8e4c9fbafedaafc91ea951" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 33365 } } peers { permanent_uuid: "6eb81312874d421c9884f64eff92eae6" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 43865 } }
I20260923 23:14:46.078287 384 raft_consensus.cc:374] T c88663338a774085b8e9fccba5b86750 P 85902a6195e544b7a8e8794e0b7c7ca1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:14:46.078329 384 raft_consensus.cc:729] T c88663338a774085b8e9fccba5b86750 P 85902a6195e544b7a8e8794e0b7c7ca1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85902a6195e544b7a8e8794e0b7c7ca1, State: Initialized, Role: FOLLOWER
I20260923 23:14:46.078434 384 consensus_queue.cc:260] T c88663338a774085b8e9fccba5b86750 P 85902a6195e544b7a8e8794e0b7c7ca1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85902a6195e544b7a8e8794e0b7c7ca1" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 42775 } } peers { permanent_uuid: "99b029603b8e4c9fbafedaafc91ea951" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 33365 } } peers { permanent_uuid: "6eb81312874d421c9884f64eff92eae6" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 43865 } }
I20260923 23:14:46.078686 384 ts_tablet_manager.cc:1428] T c88663338a774085b8e9fccba5b86750 P 85902a6195e544b7a8e8794e0b7c7ca1: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260923 23:14:46.078814 32546 heartbeater.cc:503] Master 127.24.227.254:41951 was elected leader, sending a full tablet report...
I20260923 23:14:46.079206 385 raft_consensus.cc:348] T c88663338a774085b8e9fccba5b86750 P 99b029603b8e4c9fbafedaafc91ea951 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85902a6195e544b7a8e8794e0b7c7ca1" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 42775 } } peers { permanent_uuid: "99b029603b8e4c9fbafedaafc91ea951" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 33365 } } peers { permanent_uuid: "6eb81312874d421c9884f64eff92eae6" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 43865 } }
I20260923 23:14:46.079358 385 raft_consensus.cc:374] T c88663338a774085b8e9fccba5b86750 P 99b029603b8e4c9fbafedaafc91ea951 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:14:46.079402 385 raft_consensus.cc:729] T c88663338a774085b8e9fccba5b86750 P 99b029603b8e4c9fbafedaafc91ea951 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99b029603b8e4c9fbafedaafc91ea951, State: Initialized, Role: FOLLOWER
I20260923 23:14:46.079502 385 consensus_queue.cc:260] T c88663338a774085b8e9fccba5b86750 P 99b029603b8e4c9fbafedaafc91ea951 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85902a6195e544b7a8e8794e0b7c7ca1" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 42775 } } peers { permanent_uuid: "99b029603b8e4c9fbafedaafc91ea951" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 33365 } } peers { permanent_uuid: "6eb81312874d421c9884f64eff92eae6" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 43865 } }
I20260923 23:14:46.079746 385 ts_tablet_manager.cc:1428] T c88663338a774085b8e9fccba5b86750 P 99b029603b8e4c9fbafedaafc91ea951: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260923 23:14:46.079824 32682 heartbeater.cc:503] Master 127.24.227.254:41951 was elected leader, sending a full tablet report...
I20260923 23:14:46.080474 389 tablet_bootstrap.cc:492] T c88663338a774085b8e9fccba5b86750 P 6eb81312874d421c9884f64eff92eae6: Bootstrap starting.
I20260923 23:14:46.081661 389 tablet_bootstrap.cc:654] T c88663338a774085b8e9fccba5b86750 P 6eb81312874d421c9884f64eff92eae6: Neither blocks nor log segments found. Creating new log.
I20260923 23:14:46.081969 389 log.cc:824] T c88663338a774085b8e9fccba5b86750 P 6eb81312874d421c9884f64eff92eae6: Log is configured to *not* fsync() on all Append() calls
I20260923 23:14:46.082517 389 tablet_bootstrap.cc:492] T c88663338a774085b8e9fccba5b86750 P 6eb81312874d421c9884f64eff92eae6: No bootstrap required, opened a new log
I20260923 23:14:46.082579 389 ts_tablet_manager.cc:1397] T c88663338a774085b8e9fccba5b86750 P 6eb81312874d421c9884f64eff92eae6: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260923 23:14:46.084241 389 raft_consensus.cc:348] T c88663338a774085b8e9fccba5b86750 P 6eb81312874d421c9884f64eff92eae6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85902a6195e544b7a8e8794e0b7c7ca1" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 42775 } } peers { permanent_uuid: "99b029603b8e4c9fbafedaafc91ea951" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 33365 } } peers { permanent_uuid: "6eb81312874d421c9884f64eff92eae6" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 43865 } }
I20260923 23:14:46.084348 389 raft_consensus.cc:374] T c88663338a774085b8e9fccba5b86750 P 6eb81312874d421c9884f64eff92eae6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:14:46.084367 389 raft_consensus.cc:729] T c88663338a774085b8e9fccba5b86750 P 6eb81312874d421c9884f64eff92eae6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6eb81312874d421c9884f64eff92eae6, State: Initialized, Role: FOLLOWER
I20260923 23:14:46.084465 389 consensus_queue.cc:260] T c88663338a774085b8e9fccba5b86750 P 6eb81312874d421c9884f64eff92eae6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85902a6195e544b7a8e8794e0b7c7ca1" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 42775 } } peers { permanent_uuid: "99b029603b8e4c9fbafedaafc91ea951" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 33365 } } peers { permanent_uuid: "6eb81312874d421c9884f64eff92eae6" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 43865 } }
I20260923 23:14:46.084717 389 ts_tablet_manager.cc:1428] T c88663338a774085b8e9fccba5b86750 P 6eb81312874d421c9884f64eff92eae6: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260923 23:14:46.084753 350 heartbeater.cc:503] Master 127.24.227.254:41951 was elected leader, sending a full tablet report...
W20260923 23:14:46.155038 32684 tablet.cc:2369] T c88663338a774085b8e9fccba5b86750 P 99b029603b8e4c9fbafedaafc91ea951: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 23:14:46.272162 388 raft_consensus.cc:482] T c88663338a774085b8e9fccba5b86750 P 85902a6195e544b7a8e8794e0b7c7ca1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 23:14:46.272270 388 raft_consensus.cc:504] T c88663338a774085b8e9fccba5b86750 P 85902a6195e544b7a8e8794e0b7c7ca1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85902a6195e544b7a8e8794e0b7c7ca1" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 42775 } } peers { permanent_uuid: "99b029603b8e4c9fbafedaafc91ea951" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 33365 } } peers { permanent_uuid: "6eb81312874d421c9884f64eff92eae6" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 43865 } }
I20260923 23:14:46.272648 388 leader_election.cc:288] T c88663338a774085b8e9fccba5b86750 P 85902a6195e544b7a8e8794e0b7c7ca1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99b029603b8e4c9fbafedaafc91ea951 (127.24.227.194:33365), 6eb81312874d421c9884f64eff92eae6 (127.24.227.195:43865)
I20260923 23:14:46.275408 32550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:46.272843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.194:33365 (local address 127.24.227.193:32969)
0923 23:14:46.272912 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:46.272922 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:46.272995 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:46.273139 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:46.273140 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:46.273148 (+ 8us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:14:46.273224 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:46.273226 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:46.273848 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:46.273849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:46.274584 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:46.274586 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:46.275134 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:46.275135 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:46.275218 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:46.275221 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 23:14:46.275286 (+ 65us) client_negotiation.cc:240] Negotiation successful
0923 23:14:46.275330 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50,"mutex_wait_us":3}
I20260923 23:14:46.275398 383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:46.272917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.193:32969 (local address 127.24.227.194:33365)
0923 23:14:46.272978 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:46.272980 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:46.272989 (+ 9us) server_negotiation.cc:407] Connection header received
0923 23:14:46.273061 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:46.273062 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:46.273071 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:46.273138 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:14:46.273291 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:46.273757 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:46.274692 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:46.275056 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:46.275139 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:46.275147 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:46.275291 (+ 144us) server_negotiation.cc:299] Negotiation successful
0923 23:14:46.275330 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260923 23:14:46.275426 32549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:46.272991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.195:43865 (local address 127.24.227.193:50353)
0923 23:14:46.273034 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:46.273043 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:46.273082 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:46.273261 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:46.273262 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:46.273263 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:14:46.273326 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:46.273327 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:46.273919 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:46.273920 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:46.274627 (+ 707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:46.274629 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:46.275148 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:46.275149 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:46.275233 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:46.275235 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 23:14:46.275296 (+ 61us) client_negotiation.cc:240] Negotiation successful
0923 23:14:46.275338 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26,"mutex_wait_us":3}
I20260923 23:14:46.275416 382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:46.273096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.193:50353 (local address 127.24.227.195:43865)
0923 23:14:46.273144 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:46.273151 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:46.273158 (+ 7us) server_negotiation.cc:407] Connection header received
0923 23:14:46.273181 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:46.273181 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:46.273188 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:46.273246 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:14:46.273398 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:46.273841 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:46.274720 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:46.275058 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:46.275148 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:46.275155 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:46.275314 (+ 159us) server_negotiation.cc:299] Negotiation successful
0923 23:14:46.275351 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260923 23:14:46.275779 303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c88663338a774085b8e9fccba5b86750" candidate_uuid: "85902a6195e544b7a8e8794e0b7c7ca1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6eb81312874d421c9884f64eff92eae6" is_pre_election: true
I20260923 23:14:46.275761 32635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c88663338a774085b8e9fccba5b86750" candidate_uuid: "85902a6195e544b7a8e8794e0b7c7ca1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99b029603b8e4c9fbafedaafc91ea951" is_pre_election: true
I20260923 23:14:46.275954 303 raft_consensus.cc:2454] T c88663338a774085b8e9fccba5b86750 P 6eb81312874d421c9884f64eff92eae6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85902a6195e544b7a8e8794e0b7c7ca1 in term 0.
I20260923 23:14:46.275954 32635 raft_consensus.cc:2454] T c88663338a774085b8e9fccba5b86750 P 99b029603b8e4c9fbafedaafc91ea951 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85902a6195e544b7a8e8794e0b7c7ca1 in term 0.
I20260923 23:14:46.276180 32432 leader_election.cc:302] T c88663338a774085b8e9fccba5b86750 P 85902a6195e544b7a8e8794e0b7c7ca1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6eb81312874d421c9884f64eff92eae6, 85902a6195e544b7a8e8794e0b7c7ca1; no voters:
I20260923 23:14:46.276341 388 raft_consensus.cc:2781] T c88663338a774085b8e9fccba5b86750 P 85902a6195e544b7a8e8794e0b7c7ca1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 23:14:46.276405 388 raft_consensus.cc:482] T c88663338a774085b8e9fccba5b86750 P 85902a6195e544b7a8e8794e0b7c7ca1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 23:14:46.276422 388 raft_consensus.cc:3037] T c88663338a774085b8e9fccba5b86750 P 85902a6195e544b7a8e8794e0b7c7ca1 [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:14:46.277119 388 raft_consensus.cc:504] T c88663338a774085b8e9fccba5b86750 P 85902a6195e544b7a8e8794e0b7c7ca1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85902a6195e544b7a8e8794e0b7c7ca1" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 42775 } } peers { permanent_uuid: "99b029603b8e4c9fbafedaafc91ea951" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 33365 } } peers { permanent_uuid: "6eb81312874d421c9884f64eff92eae6" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 43865 } }
I20260923 23:14:46.277240 388 leader_election.cc:288] T c88663338a774085b8e9fccba5b86750 P 85902a6195e544b7a8e8794e0b7c7ca1 [CANDIDATE]: Term 1 election: Requested vote from peers 99b029603b8e4c9fbafedaafc91ea951 (127.24.227.194:33365), 6eb81312874d421c9884f64eff92eae6 (127.24.227.195:43865)
I20260923 23:14:46.277401 32635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c88663338a774085b8e9fccba5b86750" candidate_uuid: "85902a6195e544b7a8e8794e0b7c7ca1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99b029603b8e4c9fbafedaafc91ea951"
I20260923 23:14:46.277436 303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c88663338a774085b8e9fccba5b86750" candidate_uuid: "85902a6195e544b7a8e8794e0b7c7ca1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6eb81312874d421c9884f64eff92eae6"
I20260923 23:14:46.277472 32635 raft_consensus.cc:3037] T c88663338a774085b8e9fccba5b86750 P 99b029603b8e4c9fbafedaafc91ea951 [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:14:46.277515 303 raft_consensus.cc:3037] T c88663338a774085b8e9fccba5b86750 P 6eb81312874d421c9884f64eff92eae6 [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:14:46.278115 32635 raft_consensus.cc:2454] T c88663338a774085b8e9fccba5b86750 P 99b029603b8e4c9fbafedaafc91ea951 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85902a6195e544b7a8e8794e0b7c7ca1 in term 1.
I20260923 23:14:46.278259 303 raft_consensus.cc:2454] T c88663338a774085b8e9fccba5b86750 P 6eb81312874d421c9884f64eff92eae6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85902a6195e544b7a8e8794e0b7c7ca1 in term 1.
I20260923 23:14:46.278263 32432 leader_election.cc:302] T c88663338a774085b8e9fccba5b86750 P 85902a6195e544b7a8e8794e0b7c7ca1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85902a6195e544b7a8e8794e0b7c7ca1, 99b029603b8e4c9fbafedaafc91ea951; no voters:
I20260923 23:14:46.278455 388 raft_consensus.cc:2781] T c88663338a774085b8e9fccba5b86750 P 85902a6195e544b7a8e8794e0b7c7ca1 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 23:14:46.278586 388 raft_consensus.cc:686] T c88663338a774085b8e9fccba5b86750 P 85902a6195e544b7a8e8794e0b7c7ca1 [term 1 LEADER]: Becoming Leader. State: Replica: 85902a6195e544b7a8e8794e0b7c7ca1, State: Running, Role: LEADER
I20260923 23:14:46.278685 388 consensus_queue.cc:237] T c88663338a774085b8e9fccba5b86750 P 85902a6195e544b7a8e8794e0b7c7ca1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85902a6195e544b7a8e8794e0b7c7ca1" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 42775 } } peers { permanent_uuid: "99b029603b8e4c9fbafedaafc91ea951" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 33365 } } peers { permanent_uuid: "6eb81312874d421c9884f64eff92eae6" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 43865 } }
I20260923 23:14:46.279567 31572 catalog_manager.cc:5495] T c88663338a774085b8e9fccba5b86750 P 85902a6195e544b7a8e8794e0b7c7ca1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 85902a6195e544b7a8e8794e0b7c7ca1 (127.24.227.193). New cstate: current_term: 1 leader_uuid: "85902a6195e544b7a8e8794e0b7c7ca1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85902a6195e544b7a8e8794e0b7c7ca1" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 42775 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "99b029603b8e4c9fbafedaafc91ea951" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 33365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6eb81312874d421c9884f64eff92eae6" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 43865 } health_report { overall_health: UNKNOWN } } }
W20260923 23:14:46.285053 32548 tablet.cc:2369] T c88663338a774085b8e9fccba5b86750 P 85902a6195e544b7a8e8794e0b7c7ca1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 23:14:46.291477 352 tablet.cc:2369] T c88663338a774085b8e9fccba5b86750 P 6eb81312874d421c9884f64eff92eae6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 23:14:46.293535 380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:46.291038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46048 (local address 127.24.227.193:42775)
0923 23:14:46.291086 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:46.291088 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:46.291109 (+ 21us) server_negotiation.cc:407] Connection header received
0923 23:14:46.291171 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:46.291172 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:46.291192 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:46.291275 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 23:14:46.291441 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:46.292015 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:46.292746 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:46.292950 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:46.293029 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:46.293217 (+ 188us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 23:14:46.293308 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 23:14:46.293409 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:46.293449 (+ 40us) server_negotiation.cc:299] Negotiation successful
0923 23:14:46.293468 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260923 23:14:46.295302 303 raft_consensus.cc:1261] T c88663338a774085b8e9fccba5b86750 P 6eb81312874d421c9884f64eff92eae6 [term 1 FOLLOWER]: Refusing update from remote peer 85902a6195e544b7a8e8794e0b7c7ca1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 23:14:46.295624 32635 raft_consensus.cc:1261] T c88663338a774085b8e9fccba5b86750 P 99b029603b8e4c9fbafedaafc91ea951 [term 1 FOLLOWER]: Refusing update from remote peer 85902a6195e544b7a8e8794e0b7c7ca1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 23:14:46.295619 388 consensus_queue.cc:1035] T c88663338a774085b8e9fccba5b86750 P 85902a6195e544b7a8e8794e0b7c7ca1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6eb81312874d421c9884f64eff92eae6" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 43865 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 23:14:46.295986 388 consensus_queue.cc:1035] T c88663338a774085b8e9fccba5b86750 P 85902a6195e544b7a8e8794e0b7c7ca1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99b029603b8e4c9fbafedaafc91ea951" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 33365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 23:14:46.298915 396 mvcc.cc:205] Tried to move back new op lower bound from 7332680852662956032 to 7332680852597501952. Current Snapshot: MvccSnapshot[applied={T|T < 7332680852662956032}]
I20260923 23:14:46.299803 397 mvcc.cc:205] Tried to move back new op lower bound from 7332680852662956032 to 7332680852597501952. Current Snapshot: MvccSnapshot[applied={T|T < 7332680852662956032}]
I20260923 23:14:46.300503 399 mvcc.cc:205] Tried to move back new op lower bound from 7332680852662956032 to 7332680852597501952. Current Snapshot: MvccSnapshot[applied={T|T < 7332680852662956032 or (T in {7332680852662956032})}]
I20260923 23:14:46.301321 31573 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35700:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 23:14:46.301407 31573 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35700:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 23:14:46.302591 31573 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2ddae489e6844395a47c75343be04fa5: Sending DeleteTablet for 3 replicas of tablet c88663338a774085b8e9fccba5b86750
I20260923 23:14:46.303761 32615 tablet_service.cc:1514] Processing DeleteTablet for tablet c88663338a774085b8e9fccba5b86750 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 23:14:46 UTC) from {username='oryx', principal='oryx/127.24.227.254@KRBTEST.COM'} at 127.0.0.1:35268
I20260923 23:14:46.303759 32479 tablet_service.cc:1514] Processing DeleteTablet for tablet c88663338a774085b8e9fccba5b86750 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 23:14:46 UTC) from {username='oryx', principal='oryx/127.24.227.254@KRBTEST.COM'} at 127.0.0.1:46032
I20260923 23:14:46.304009 32751 tablet_service.cc:1514] Processing DeleteTablet for tablet c88663338a774085b8e9fccba5b86750 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 23:14:46 UTC) from {username='oryx', principal='oryx/127.24.227.254@KRBTEST.COM'} at 127.0.0.1:48976
I20260923 23:14:46.304036 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 31541
I20260923 23:14:46.304239 407 tablet_replica.cc:330] T c88663338a774085b8e9fccba5b86750 P 99b029603b8e4c9fbafedaafc91ea951: stopping tablet replica
I20260923 23:14:46.304239 408 tablet_replica.cc:330] T c88663338a774085b8e9fccba5b86750 P 6eb81312874d421c9884f64eff92eae6: stopping tablet replica
I20260923 23:14:46.304338 407 raft_consensus.cc:2229] T c88663338a774085b8e9fccba5b86750 P 99b029603b8e4c9fbafedaafc91ea951 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 23:14:46.304428 408 raft_consensus.cc:2229] T c88663338a774085b8e9fccba5b86750 P 6eb81312874d421c9884f64eff92eae6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 23:14:46.304556 407 raft_consensus.cc:2258] T c88663338a774085b8e9fccba5b86750 P 99b029603b8e4c9fbafedaafc91ea951 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 23:14:46.304556 408 raft_consensus.cc:2258] T c88663338a774085b8e9fccba5b86750 P 6eb81312874d421c9884f64eff92eae6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 23:14:46.305651 408 ts_tablet_manager.cc:1905] T c88663338a774085b8e9fccba5b86750 P 6eb81312874d421c9884f64eff92eae6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 23:14:46.307018 407 ts_tablet_manager.cc:1905] T c88663338a774085b8e9fccba5b86750 P 99b029603b8e4c9fbafedaafc91ea951: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 23:14:46.307251 408 ts_tablet_manager.cc:1918] T c88663338a774085b8e9fccba5b86750 P 6eb81312874d421c9884f64eff92eae6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 23:14:46.307366 408 log.cc:1196] T c88663338a774085b8e9fccba5b86750 P 6eb81312874d421c9884f64eff92eae6: Deleting WAL directory at /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/ts-2/wal/wals/c88663338a774085b8e9fccba5b86750
I20260923 23:14:46.307689 408 ts_tablet_manager.cc:1939] T c88663338a774085b8e9fccba5b86750 P 6eb81312874d421c9884f64eff92eae6: Deleting consensus metadata
I20260923 23:14:46.308557 407 ts_tablet_manager.cc:1918] T c88663338a774085b8e9fccba5b86750 P 99b029603b8e4c9fbafedaafc91ea951: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 23:14:46.308640 407 log.cc:1196] T c88663338a774085b8e9fccba5b86750 P 99b029603b8e4c9fbafedaafc91ea951: Deleting WAL directory at /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/ts-1/wal/wals/c88663338a774085b8e9fccba5b86750
I20260923 23:14:46.308857 407 ts_tablet_manager.cc:1939] T c88663338a774085b8e9fccba5b86750 P 99b029603b8e4c9fbafedaafc91ea951: Deleting consensus metadata
W20260923 23:14:46.311509 32706 connection.cc:535] server connection from 127.0.0.1:48976 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260923 23:14:46.312063 406 tablet_replica.cc:330] T c88663338a774085b8e9fccba5b86750 P 85902a6195e544b7a8e8794e0b7c7ca1: stopping tablet replica
I20260923 23:14:46.312219 406 raft_consensus.cc:2229] T c88663338a774085b8e9fccba5b86750 P 85902a6195e544b7a8e8794e0b7c7ca1 [term 1 LEADER]: Raft consensus shutting down.
I20260923 23:14:46.312335 406 raft_consensus.cc:2258] T c88663338a774085b8e9fccba5b86750 P 85902a6195e544b7a8e8794e0b7c7ca1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 23:14:46.312731 406 ts_tablet_manager.cc:1905] T c88663338a774085b8e9fccba5b86750 P 85902a6195e544b7a8e8794e0b7c7ca1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 23:14:46.314260 406 ts_tablet_manager.cc:1918] T c88663338a774085b8e9fccba5b86750 P 85902a6195e544b7a8e8794e0b7c7ca1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 23:14:46.314327 406 log.cc:1196] T c88663338a774085b8e9fccba5b86750 P 85902a6195e544b7a8e8794e0b7c7ca1: Deleting WAL directory at /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/ts-0/wal/wals/c88663338a774085b8e9fccba5b86750
I20260923 23:14:46.314567 406 ts_tablet_manager.cc:1939] T c88663338a774085b8e9fccba5b86750 P 85902a6195e544b7a8e8794e0b7c7ca1: Deleting consensus metadata
W20260923 23:14:46.314915 32433 connection.cc:441] server connection from 127.0.0.1:46032 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46032 (request call id 1) could send its response
I20260923 23:14:46.323541 364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:46.315660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41606 (local address 127.24.227.252:46859)
0923 23:14:46.315702 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:46.315704 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:46.315894 (+ 190us) server_negotiation.cc:407] Connection header received
0923 23:14:46.315940 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:46.315942 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:46.315956 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:46.315987 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:46.317112 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:46.317855 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:46.318716 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:46.318999 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:46.319062 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:46.320489 (+ 1427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:46.320490 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:46.320490 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:46.320496 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:46.322358 (+ 1862us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:46.322721 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:46.322722 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:46.322723 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:46.322774 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:46.322994 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:46.322995 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:46.322996 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:46.323124 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:46.323229 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:46.323365 (+ 136us) server_negotiation.cc:299] Negotiation successful
0923 23:14:46.323449 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260923 23:14:46.323999 363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:46.315432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50662 (local address 127.24.227.253:35161)
0923 23:14:46.315474 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:46.315476 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:46.315614 (+ 138us) server_negotiation.cc:407] Connection header received
0923 23:14:46.315702 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:46.315704 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:46.315718 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:46.315783 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:46.316484 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:46.317720 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:46.318836 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:46.319113 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:46.319183 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:46.320442 (+ 1259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:46.320443 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:46.320444 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:46.320451 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:46.322577 (+ 2126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:46.322965 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:46.322966 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:46.322967 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:46.323027 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:46.323243 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:46.323244 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:46.323245 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:46.323376 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:46.323447 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:46.323777 (+ 330us) server_negotiation.cc:299] Negotiation successful
0923 23:14:46.323861 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 23:14:47.300393 415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:47.300099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41951 (local address 127.24.227.193:53255)
0923 23:14:47.300230 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:47.300314 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:41951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":59,"threads_started":1}
W20260923 23:14:47.300668 32546 heartbeater.cc:644] Failed to heartbeat to 127.24.227.254:41951 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.24.227.254:41951: connect: Connection refused (error 111)
I20260923 23:14:47.300858 416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:47.300635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41951 (local address 127.24.227.194:59185)
0923 23:14:47.300739 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:47.300812 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:41951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85,"thread_start_us":54,"threads_started":1}
W20260923 23:14:47.301196 32682 heartbeater.cc:644] Failed to heartbeat to 127.24.227.254:41951 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.24.227.254:41951: connect: Connection refused (error 111)
I20260923 23:14:47.301750 417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:47.301594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41951 (local address 127.24.227.195:59185)
0923 23:14:47.301673 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:47.301717 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:41951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68,"thread_start_us":35,"threads_started":1}
W20260923 23:14:47.301928 350 heartbeater.cc:644] Failed to heartbeat to 127.24.227.254:41951 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.24.227.254:41951: connect: Connection refused (error 111)
I20260923 23:14:47.803433 418 raft_consensus.cc:482] T 00000000000000000000000000000000 P f014784de6dd46128a093784ebbe68fb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2ddae489e6844395a47c75343be04fa5)
I20260923 23:14:47.803625 419 raft_consensus.cc:482] T 00000000000000000000000000000000 P bd675df8e6274c688bb00c0f39e018c1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2ddae489e6844395a47c75343be04fa5)
I20260923 23:14:47.803593 418 raft_consensus.cc:504] T 00000000000000000000000000000000 P f014784de6dd46128a093784ebbe68fb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "2ddae489e6844395a47c75343be04fa5" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41951 } } peers { permanent_uuid: "bd675df8e6274c688bb00c0f39e018c1" member_type: VOTER last_known_addr { host: "127.24.227.253" port: 35161 } attrs { promote: false } } peers { permanent_uuid: "f014784de6dd46128a093784ebbe68fb" member_type: VOTER last_known_addr { host: "127.24.227.252" port: 46859 } attrs { promote: false } }
I20260923 23:14:47.803726 419 raft_consensus.cc:504] T 00000000000000000000000000000000 P bd675df8e6274c688bb00c0f39e018c1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "2ddae489e6844395a47c75343be04fa5" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41951 } } peers { permanent_uuid: "bd675df8e6274c688bb00c0f39e018c1" member_type: VOTER last_known_addr { host: "127.24.227.253" port: 35161 } attrs { promote: false } } peers { permanent_uuid: "f014784de6dd46128a093784ebbe68fb" member_type: VOTER last_known_addr { host: "127.24.227.252" port: 46859 } attrs { promote: false } }
I20260923 23:14:47.804009 418 leader_election.cc:288] T 00000000000000000000000000000000 P f014784de6dd46128a093784ebbe68fb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2ddae489e6844395a47c75343be04fa5 (127.24.227.254:41951), bd675df8e6274c688bb00c0f39e018c1 (127.24.227.253:35161)
I20260923 23:14:47.804252 419 leader_election.cc:288] T 00000000000000000000000000000000 P bd675df8e6274c688bb00c0f39e018c1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2ddae489e6844395a47c75343be04fa5 (127.24.227.254:41951), f014784de6dd46128a093784ebbe68fb (127.24.227.252:46859)
I20260923 23:14:47.804399 420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:47.804090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41951 (local address 127.0.0.1:36170)
0923 23:14:47.804294 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:47.804341 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:41951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":50,"threads_started":1}
I20260923 23:14:47.804519 422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:47.804094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41951 (local address 127.0.0.1:36172)
0923 23:14:47.804283 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:47.804397 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:41951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":43,"threads_started":1}
W20260923 23:14:47.804606 32348 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.227.254:41951: connect: Connection refused (error 111)
W20260923 23:14:47.804636 32161 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.227.254:41951: connect: Connection refused (error 111)
I20260923 23:14:47.805008 422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:47.804899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41951 (local address 127.0.0.1:36192)
0923 23:14:47.804953 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:47.804982 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:41951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260923 23:14:47.805060 420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:47.804937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41951 (local address 127.0.0.1:36200)
0923 23:14:47.804989 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:47.805012 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:41951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260923 23:14:47.805147 32348 leader_election.cc:334] T 00000000000000000000000000000000 P f014784de6dd46128a093784ebbe68fb [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 2ddae489e6844395a47c75343be04fa5 (127.24.227.254:41951): Network error: Client connection negotiation failed: client connection to 127.24.227.254:41951: connect: Connection refused (error 111)
W20260923 23:14:47.805198 32161 leader_election.cc:334] T 00000000000000000000000000000000 P bd675df8e6274c688bb00c0f39e018c1 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 2ddae489e6844395a47c75343be04fa5 (127.24.227.254:41951): Network error: Client connection negotiation failed: client connection to 127.24.227.254:41951: connect: Connection refused (error 111)
I20260923 23:14:47.807693 423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:47.804273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.253:35161 (local address 127.0.0.1:50664)
0923 23:14:47.804344 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:47.804354 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:47.804406 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:47.804953 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:47.804954 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:47.804957 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:14:47.805012 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:47.805014 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:47.805822 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:47.805823 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:47.806955 (+ 1132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:47.806957 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:47.807450 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:47.807451 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:47.807503 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:47.807510 (+ 7us) client_negotiation.cc:769] Sending connection context
0923 23:14:47.807571 (+ 61us) client_negotiation.cc:240] Negotiation successful
0923 23:14:47.807609 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61,"thread_start_us":44,"threads_started":1}
I20260923 23:14:47.807693 425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:47.804476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41610 (local address 127.24.227.252:46859)
0923 23:14:47.804571 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:47.804572 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:47.804690 (+ 118us) server_negotiation.cc:407] Connection header received
0923 23:14:47.804778 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:47.804779 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:47.804789 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:47.804830 (+ 41us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:14:47.805102 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:47.805751 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:47.807027 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:47.807356 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:47.807448 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:47.807456 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:47.807590 (+ 134us) server_negotiation.cc:299] Negotiation successful
0923 23:14:47.807621 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71,"thread_start_us":42,"threads_started":1}
I20260923 23:14:47.807711 424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:47.804382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50664 (local address 127.24.227.253:35161)
0923 23:14:47.804791 (+ 409us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:47.804793 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:47.804802 (+ 9us) server_negotiation.cc:407] Connection header received
0923 23:14:47.804824 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:47.804824 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:47.804835 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:47.805095 (+ 260us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:14:47.805134 (+ 39us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:47.805752 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:47.807063 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:47.807373 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:47.807452 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:47.807460 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:47.807576 (+ 116us) server_negotiation.cc:299] Negotiation successful
0923 23:14:47.807618 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":40,"threads_started":1}
I20260923 23:14:47.807724 421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:47.804175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.252:46859 (local address 127.0.0.1:41610)
0923 23:14:47.804637 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:47.804646 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:47.804713 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:47.804897 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:47.804898 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:47.804900 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:14:47.804991 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:47.804993 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:47.805857 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:47.805857 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:47.806964 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:47.806966 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:47.807444 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:47.807446 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:47.807510 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:47.807513 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 23:14:47.807583 (+ 70us) client_negotiation.cc:240] Negotiation successful
0923 23:14:47.807622 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":446,"mutex_wait_us":4,"thread_start_us":45,"threads_started":1}
I20260923 23:14:47.807917 32187 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f014784de6dd46128a093784ebbe68fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "bd675df8e6274c688bb00c0f39e018c1" is_pre_election: true
I20260923 23:14:47.808068 32187 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bd675df8e6274c688bb00c0f39e018c1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f014784de6dd46128a093784ebbe68fb in term 1.
I20260923 23:14:47.808157 32374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bd675df8e6274c688bb00c0f39e018c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "f014784de6dd46128a093784ebbe68fb" is_pre_election: true
I20260923 23:14:47.808238 32349 leader_election.cc:302] T 00000000000000000000000000000000 P f014784de6dd46128a093784ebbe68fb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bd675df8e6274c688bb00c0f39e018c1, f014784de6dd46128a093784ebbe68fb; no voters: 2ddae489e6844395a47c75343be04fa5
I20260923 23:14:47.808261 32374 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f014784de6dd46128a093784ebbe68fb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd675df8e6274c688bb00c0f39e018c1 in term 1.
I20260923 23:14:47.808338 418 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f014784de6dd46128a093784ebbe68fb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260923 23:14:47.808375 418 raft_consensus.cc:482] T 00000000000000000000000000000000 P f014784de6dd46128a093784ebbe68fb [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2ddae489e6844395a47c75343be04fa5)
I20260923 23:14:47.808385 32163 leader_election.cc:302] T 00000000000000000000000000000000 P bd675df8e6274c688bb00c0f39e018c1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bd675df8e6274c688bb00c0f39e018c1, f014784de6dd46128a093784ebbe68fb; no voters: 2ddae489e6844395a47c75343be04fa5
I20260923 23:14:47.808404 418 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f014784de6dd46128a093784ebbe68fb [term 1 FOLLOWER]: Advancing to term 2
I20260923 23:14:47.808491 419 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bd675df8e6274c688bb00c0f39e018c1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260923 23:14:47.808529 419 raft_consensus.cc:482] T 00000000000000000000000000000000 P bd675df8e6274c688bb00c0f39e018c1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2ddae489e6844395a47c75343be04fa5)
I20260923 23:14:47.808553 419 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bd675df8e6274c688bb00c0f39e018c1 [term 1 FOLLOWER]: Advancing to term 2
I20260923 23:14:47.809294 418 raft_consensus.cc:504] T 00000000000000000000000000000000 P f014784de6dd46128a093784ebbe68fb [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "2ddae489e6844395a47c75343be04fa5" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41951 } } peers { permanent_uuid: "bd675df8e6274c688bb00c0f39e018c1" member_type: VOTER last_known_addr { host: "127.24.227.253" port: 35161 } attrs { promote: false } } peers { permanent_uuid: "f014784de6dd46128a093784ebbe68fb" member_type: VOTER last_known_addr { host: "127.24.227.252" port: 46859 } attrs { promote: false } }
I20260923 23:14:47.809334 419 raft_consensus.cc:504] T 00000000000000000000000000000000 P bd675df8e6274c688bb00c0f39e018c1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "2ddae489e6844395a47c75343be04fa5" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41951 } } peers { permanent_uuid: "bd675df8e6274c688bb00c0f39e018c1" member_type: VOTER last_known_addr { host: "127.24.227.253" port: 35161 } attrs { promote: false } } peers { permanent_uuid: "f014784de6dd46128a093784ebbe68fb" member_type: VOTER last_known_addr { host: "127.24.227.252" port: 46859 } attrs { promote: false } }
I20260923 23:14:47.809435 418 leader_election.cc:288] T 00000000000000000000000000000000 P f014784de6dd46128a093784ebbe68fb [CANDIDATE]: Term 2 election: Requested vote from peers 2ddae489e6844395a47c75343be04fa5 (127.24.227.254:41951), bd675df8e6274c688bb00c0f39e018c1 (127.24.227.253:35161)
I20260923 23:14:47.809437 419 leader_election.cc:288] T 00000000000000000000000000000000 P bd675df8e6274c688bb00c0f39e018c1 [CANDIDATE]: Term 2 election: Requested vote from peers 2ddae489e6844395a47c75343be04fa5 (127.24.227.254:41951), f014784de6dd46128a093784ebbe68fb (127.24.227.252:46859)
I20260923 23:14:47.809652 423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:47.809550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41951 (local address 127.0.0.1:36204)
0923 23:14:47.809580 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:47.809605 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:41951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
I20260923 23:14:47.809732 32374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bd675df8e6274c688bb00c0f39e018c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "f014784de6dd46128a093784ebbe68fb"
I20260923 23:14:47.809741 32187 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f014784de6dd46128a093784ebbe68fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "bd675df8e6274c688bb00c0f39e018c1"
I20260923 23:14:47.809774 421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:47.809550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41951 (local address 127.0.0.1:36202)
0923 23:14:47.809729 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:47.809755 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:41951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166}
I20260923 23:14:47.809811 32374 raft_consensus.cc:2379] T 00000000000000000000000000000000 P f014784de6dd46128a093784ebbe68fb [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate bd675df8e6274c688bb00c0f39e018c1 in current term 2: Already voted for candidate f014784de6dd46128a093784ebbe68fb in this term.
I20260923 23:14:47.809824 32187 raft_consensus.cc:2379] T 00000000000000000000000000000000 P bd675df8e6274c688bb00c0f39e018c1 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate f014784de6dd46128a093784ebbe68fb in current term 2: Already voted for candidate bd675df8e6274c688bb00c0f39e018c1 in this term.
I20260923 23:14:47.810076 421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:47.809990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41951 (local address 127.0.0.1:36220)
0923 23:14:47.810029 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:47.810051 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:41951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260923 23:14:47.810108 423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:47.810035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41951 (local address 127.0.0.1:36230)
0923 23:14:47.810068 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:47.810086 (+ 18us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:41951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260923 23:14:47.810149 32348 leader_election.cc:334] T 00000000000000000000000000000000 P f014784de6dd46128a093784ebbe68fb [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 2ddae489e6844395a47c75343be04fa5 (127.24.227.254:41951): Network error: Client connection negotiation failed: client connection to 127.24.227.254:41951: connect: Connection refused (error 111)
I20260923 23:14:47.810177 32348 leader_election.cc:302] T 00000000000000000000000000000000 P f014784de6dd46128a093784ebbe68fb [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f014784de6dd46128a093784ebbe68fb; no voters: 2ddae489e6844395a47c75343be04fa5, bd675df8e6274c688bb00c0f39e018c1
W20260923 23:14:47.810163 32161 leader_election.cc:334] T 00000000000000000000000000000000 P bd675df8e6274c688bb00c0f39e018c1 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 2ddae489e6844395a47c75343be04fa5 (127.24.227.254:41951): Network error: Client connection negotiation failed: client connection to 127.24.227.254:41951: connect: Connection refused (error 111)
I20260923 23:14:47.810211 32161 leader_election.cc:302] T 00000000000000000000000000000000 P bd675df8e6274c688bb00c0f39e018c1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd675df8e6274c688bb00c0f39e018c1; no voters: 2ddae489e6844395a47c75343be04fa5, f014784de6dd46128a093784ebbe68fb
I20260923 23:14:47.810256 418 raft_consensus.cc:2726] T 00000000000000000000000000000000 P f014784de6dd46128a093784ebbe68fb [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260923 23:14:47.810305 419 raft_consensus.cc:2726] T 00000000000000000000000000000000 P bd675df8e6274c688bb00c0f39e018c1 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260923 23:14:48.301246 428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:48.301006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41951 (local address 127.24.227.193:47923)
0923 23:14:48.301143 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:48.301187 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:41951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":108,"threads_started":1}
I20260923 23:14:48.301765 429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:48.301574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41951 (local address 127.24.227.194:57975)
0923 23:14:48.301689 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:48.301722 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:41951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":78,"threads_started":1}
I20260923 23:14:48.302381 430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:48.302216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41951 (local address 127.24.227.195:39371)
0923 23:14:48.302318 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:48.302348 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:41951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":53,"threads_started":1}
I20260923 23:14:49.302002 431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:49.301770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41951 (local address 127.24.227.193:58485)
0923 23:14:49.301886 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:49.301928 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:41951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":63,"threads_started":1}
I20260923 23:14:49.302289 432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:49.302072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41951 (local address 127.24.227.194:34457)
0923 23:14:49.302185 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:49.302215 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:41951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":43,"threads_started":1}
I20260923 23:14:49.302893 433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:49.302745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41951 (local address 127.24.227.195:51899)
0923 23:14:49.302834 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:49.302863 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:41951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76,"thread_start_us":43,"threads_started":1}
I20260923 23:14:49.810020 434 raft_consensus.cc:482] T 00000000000000000000000000000000 P bd675df8e6274c688bb00c0f39e018c1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 23:14:49.810091 434 raft_consensus.cc:504] T 00000000000000000000000000000000 P bd675df8e6274c688bb00c0f39e018c1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "2ddae489e6844395a47c75343be04fa5" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41951 } } peers { permanent_uuid: "bd675df8e6274c688bb00c0f39e018c1" member_type: VOTER last_known_addr { host: "127.24.227.253" port: 35161 } attrs { promote: false } } peers { permanent_uuid: "f014784de6dd46128a093784ebbe68fb" member_type: VOTER last_known_addr { host: "127.24.227.252" port: 46859 } attrs { promote: false } }
I20260923 23:14:49.810227 434 leader_election.cc:288] T 00000000000000000000000000000000 P bd675df8e6274c688bb00c0f39e018c1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2ddae489e6844395a47c75343be04fa5 (127.24.227.254:41951), f014784de6dd46128a093784ebbe68fb (127.24.227.252:46859)
I20260923 23:14:49.810465 32374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bd675df8e6274c688bb00c0f39e018c1" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "f014784de6dd46128a093784ebbe68fb" is_pre_election: true
I20260923 23:14:49.810546 32374 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f014784de6dd46128a093784ebbe68fb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd675df8e6274c688bb00c0f39e018c1 in term 2.
I20260923 23:14:49.810578 435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:49.810313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41951 (local address 127.0.0.1:47534)
0923 23:14:49.810479 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:49.810513 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:41951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":49,"threads_started":1}
I20260923 23:14:49.810694 32163 leader_election.cc:302] T 00000000000000000000000000000000 P bd675df8e6274c688bb00c0f39e018c1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd675df8e6274c688bb00c0f39e018c1, f014784de6dd46128a093784ebbe68fb; no voters:
I20260923 23:14:49.810771 434 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bd675df8e6274c688bb00c0f39e018c1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260923 23:14:49.810822 434 raft_consensus.cc:482] T 00000000000000000000000000000000 P bd675df8e6274c688bb00c0f39e018c1 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 23:14:49.810846 434 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bd675df8e6274c688bb00c0f39e018c1 [term 2 FOLLOWER]: Advancing to term 3
I20260923 23:14:49.811055 435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:49.810938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41951 (local address 127.0.0.1:47550)
0923 23:14:49.811005 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:49.811035 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:41951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260923 23:14:49.811133 32161 leader_election.cc:334] T 00000000000000000000000000000000 P bd675df8e6274c688bb00c0f39e018c1 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 2ddae489e6844395a47c75343be04fa5 (127.24.227.254:41951): Network error: Client connection negotiation failed: client connection to 127.24.227.254:41951: connect: Connection refused (error 111)
I20260923 23:14:49.811631 434 raft_consensus.cc:504] T 00000000000000000000000000000000 P bd675df8e6274c688bb00c0f39e018c1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "2ddae489e6844395a47c75343be04fa5" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41951 } } peers { permanent_uuid: "bd675df8e6274c688bb00c0f39e018c1" member_type: VOTER last_known_addr { host: "127.24.227.253" port: 35161 } attrs { promote: false } } peers { permanent_uuid: "f014784de6dd46128a093784ebbe68fb" member_type: VOTER last_known_addr { host: "127.24.227.252" port: 46859 } attrs { promote: false } }
I20260923 23:14:49.811762 434 leader_election.cc:288] T 00000000000000000000000000000000 P bd675df8e6274c688bb00c0f39e018c1 [CANDIDATE]: Term 3 election: Requested vote from peers 2ddae489e6844395a47c75343be04fa5 (127.24.227.254:41951), f014784de6dd46128a093784ebbe68fb (127.24.227.252:46859)
I20260923 23:14:49.811924 435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:49.811815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41951 (local address 127.0.0.1:47552)
0923 23:14:49.811852 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:49.811897 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:41951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260923 23:14:49.811914 32374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bd675df8e6274c688bb00c0f39e018c1" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "f014784de6dd46128a093784ebbe68fb"
I20260923 23:14:49.812016 32374 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f014784de6dd46128a093784ebbe68fb [term 2 FOLLOWER]: Advancing to term 3
I20260923 23:14:49.812258 435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:49.812182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41951 (local address 127.0.0.1:47556)
0923 23:14:49.812221 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:49.812242 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:41951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260923 23:14:49.812383 32161 leader_election.cc:334] T 00000000000000000000000000000000 P bd675df8e6274c688bb00c0f39e018c1 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 2ddae489e6844395a47c75343be04fa5 (127.24.227.254:41951): Network error: Client connection negotiation failed: client connection to 127.24.227.254:41951: connect: Connection refused (error 111)
I20260923 23:14:49.812705 32374 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f014784de6dd46128a093784ebbe68fb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd675df8e6274c688bb00c0f39e018c1 in term 3.
I20260923 23:14:49.812853 32163 leader_election.cc:302] T 00000000000000000000000000000000 P bd675df8e6274c688bb00c0f39e018c1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bd675df8e6274c688bb00c0f39e018c1, f014784de6dd46128a093784ebbe68fb; no voters: 2ddae489e6844395a47c75343be04fa5
I20260923 23:14:49.812952 434 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bd675df8e6274c688bb00c0f39e018c1 [term 3 FOLLOWER]: Leader election won for term 3
I20260923 23:14:49.813071 434 raft_consensus.cc:686] T 00000000000000000000000000000000 P bd675df8e6274c688bb00c0f39e018c1 [term 3 LEADER]: Becoming Leader. State: Replica: bd675df8e6274c688bb00c0f39e018c1, State: Running, Role: LEADER
I20260923 23:14:49.813189 434 consensus_queue.cc:237] T 00000000000000000000000000000000 P bd675df8e6274c688bb00c0f39e018c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "2ddae489e6844395a47c75343be04fa5" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41951 } } peers { permanent_uuid: "bd675df8e6274c688bb00c0f39e018c1" member_type: VOTER last_known_addr { host: "127.24.227.253" port: 35161 } attrs { promote: false } } peers { permanent_uuid: "f014784de6dd46128a093784ebbe68fb" member_type: VOTER last_known_addr { host: "127.24.227.252" port: 46859 } attrs { promote: false } }
I20260923 23:14:49.813570 437 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd675df8e6274c688bb00c0f39e018c1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bd675df8e6274c688bb00c0f39e018c1. Latest consensus state: current_term: 3 leader_uuid: "bd675df8e6274c688bb00c0f39e018c1" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "2ddae489e6844395a47c75343be04fa5" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41951 } } peers { permanent_uuid: "bd675df8e6274c688bb00c0f39e018c1" member_type: VOTER last_known_addr { host: "127.24.227.253" port: 35161 } attrs { promote: false } } peers { permanent_uuid: "f014784de6dd46128a093784ebbe68fb" member_type: VOTER last_known_addr { host: "127.24.227.252" port: 46859 } attrs { promote: false } } }
I20260923 23:14:49.813651 437 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd675df8e6274c688bb00c0f39e018c1 [sys.catalog]: This master's current role is: LEADER
I20260923 23:14:49.813794 439 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 23:14:49.814234 439 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 23:14:49.814306 439 catalog_manager.cc:1259] Loaded cluster ID: 25a76464fda54de4ba6067e47f542e90
I20260923 23:14:49.814335 439 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 23:14:49.814507 439 catalog_manager.cc:1504] Loading token signing keys...
I20260923 23:14:49.814589 439 catalog_manager.cc:5879] T 00000000000000000000000000000000 P bd675df8e6274c688bb00c0f39e018c1: Loaded TSK: 0
I20260923 23:14:49.814726 439 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 23:14:49.884595 32177 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50662:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 23:14:49.886173 32177 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 23:14:49.887140 435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:49.887035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41951 (local address 127.0.0.1:47566)
0923 23:14:49.887084 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:49.887117 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:41951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260923 23:14:49.887153 32374 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f014784de6dd46128a093784ebbe68fb [term 3 FOLLOWER]: Refusing update from remote peer bd675df8e6274c688bb00c0f39e018c1: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260923 23:14:49.887373 434 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bd675df8e6274c688bb00c0f39e018c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f014784de6dd46128a093784ebbe68fb" member_type: VOTER last_known_addr { host: "127.24.227.252" port: 46859 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260923 23:14:49.887486 435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:49.887392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41951 (local address 127.0.0.1:47570)
0923 23:14:49.887440 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:49.887469 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:41951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260923 23:14:49.887631 32161 consensus_peers.cc:487] T 00000000000000000000000000000000 P bd675df8e6274c688bb00c0f39e018c1 -> Peer 2ddae489e6844395a47c75343be04fa5 (127.24.227.254:41951): Couldn't send request to peer 2ddae489e6844395a47c75343be04fa5. Status: Network error: Client connection negotiation failed: client connection to 127.24.227.254:41951: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260923 23:14:49.887960 441 sys_catalog.cc:455] T 00000000000000000000000000000000 P f014784de6dd46128a093784ebbe68fb [sys.catalog]: SysCatalogTable state changed. Reason: New leader bd675df8e6274c688bb00c0f39e018c1. Latest consensus state: current_term: 3 leader_uuid: "bd675df8e6274c688bb00c0f39e018c1" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "2ddae489e6844395a47c75343be04fa5" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41951 } } peers { permanent_uuid: "bd675df8e6274c688bb00c0f39e018c1" member_type: VOTER last_known_addr { host: "127.24.227.253" port: 35161 } attrs { promote: false } } peers { permanent_uuid: "f014784de6dd46128a093784ebbe68fb" member_type: VOTER last_known_addr { host: "127.24.227.252" port: 46859 } attrs { promote: false } } }
I20260923 23:14:49.888065 441 sys_catalog.cc:458] T 00000000000000000000000000000000 P f014784de6dd46128a093784ebbe68fb [sys.catalog]: This master's current role is: FOLLOWER
I20260923 23:14:49.888509 434 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd675df8e6274c688bb00c0f39e018c1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "bd675df8e6274c688bb00c0f39e018c1" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "2ddae489e6844395a47c75343be04fa5" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41951 } } peers { permanent_uuid: "bd675df8e6274c688bb00c0f39e018c1" member_type: VOTER last_known_addr { host: "127.24.227.253" port: 35161 } attrs { promote: false } } peers { permanent_uuid: "f014784de6dd46128a093784ebbe68fb" member_type: VOTER last_known_addr { host: "127.24.227.252" port: 46859 } attrs { promote: false } } }
I20260923 23:14:49.888597 434 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd675df8e6274c688bb00c0f39e018c1 [sys.catalog]: This master's current role is: LEADER
I20260923 23:14:49.888557 441 sys_catalog.cc:455] T 00000000000000000000000000000000 P f014784de6dd46128a093784ebbe68fb [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "bd675df8e6274c688bb00c0f39e018c1" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "2ddae489e6844395a47c75343be04fa5" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41951 } } peers { permanent_uuid: "bd675df8e6274c688bb00c0f39e018c1" member_type: VOTER last_known_addr { host: "127.24.227.253" port: 35161 } attrs { promote: false } } peers { permanent_uuid: "f014784de6dd46128a093784ebbe68fb" member_type: VOTER last_known_addr { host: "127.24.227.252" port: 46859 } attrs { promote: false } } }
I20260923 23:14:49.888628 441 sys_catalog.cc:458] T 00000000000000000000000000000000 P f014784de6dd46128a093784ebbe68fb [sys.catalog]: This master's current role is: FOLLOWER
I20260923 23:14:49.893782 435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:49.890235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.195:43865 (local address 127.0.0.1:52208)
0923 23:14:49.890281 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:49.890291 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:49.890338 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:49.890669 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:49.890670 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:49.890672 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:14:49.890738 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:49.890740 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:49.891697 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:49.891697 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:49.892872 (+ 1175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:49.892874 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:49.893570 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:49.893570 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:49.893625 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:49.893628 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 23:14:49.893666 (+ 38us) client_negotiation.cc:240] Negotiation successful
0923 23:14:49.893705 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31,"mutex_wait_us":44}
I20260923 23:14:49.893952 448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:49.890416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54718 (local address 127.24.227.194:33365)
0923 23:14:49.890533 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:49.890535 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:49.890546 (+ 11us) server_negotiation.cc:407] Connection header received
0923 23:14:49.890574 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:49.890575 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:49.890585 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:49.890735 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:14:49.890907 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:49.891630 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:49.893194 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:49.893556 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:49.893645 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:49.893654 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:49.893822 (+ 168us) server_negotiation.cc:299] Negotiation successful
0923 23:14:49.893888 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":66,"threads_started":1}
I20260923 23:14:49.894150 446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:49.890328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.194:33365 (local address 127.0.0.1:54718)
0923 23:14:49.890458 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:49.890469 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:49.890524 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:49.890773 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:49.890774 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:49.890776 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:14:49.890823 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:49.890825 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:49.891699 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:49.891699 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:49.893116 (+ 1417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:49.893118 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:49.893645 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:49.893646 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:49.893707 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:49.893710 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 23:14:49.893813 (+ 103us) client_negotiation.cc:240] Negotiation successful
0923 23:14:49.893933 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"mutex_wait_us":61,"thread_start_us":85,"threads_started":1}
I20260923 23:14:49.894253 447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:49.890420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52208 (local address 127.24.227.195:43865)
0923 23:14:49.890529 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:49.890531 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:49.890540 (+ 9us) server_negotiation.cc:407] Connection header received
0923 23:14:49.890566 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:49.890567 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:49.890578 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:49.890798 (+ 220us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:14:49.890847 (+ 49us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:49.891609 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:49.892946 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:49.893466 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:49.893533 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:49.893541 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:49.894130 (+ 589us) server_negotiation.cc:299] Negotiation successful
0923 23:14:49.894176 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88,"thread_start_us":55,"threads_started":1}
I20260923 23:14:49.894466 32751 tablet_service.cc:1467] Processing CreateTablet for tablet b10e9ad7b61744f38e0f67a22cba3177 (DEFAULT_TABLE table=test-table [id=0d46eb4193a14b77b42effbf3d9c5c79]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 23:14:49.894577 32615 tablet_service.cc:1467] Processing CreateTablet for tablet b10e9ad7b61744f38e0f67a22cba3177 (DEFAULT_TABLE table=test-table [id=0d46eb4193a14b77b42effbf3d9c5c79]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 23:14:49.894600 32751 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b10e9ad7b61744f38e0f67a22cba3177. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:14:49.894675 32615 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b10e9ad7b61744f38e0f67a22cba3177. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:14:49.895820 451 tablet_bootstrap.cc:492] T b10e9ad7b61744f38e0f67a22cba3177 P 99b029603b8e4c9fbafedaafc91ea951: Bootstrap starting.
I20260923 23:14:49.896128 452 tablet_bootstrap.cc:492] T b10e9ad7b61744f38e0f67a22cba3177 P 6eb81312874d421c9884f64eff92eae6: Bootstrap starting.
I20260923 23:14:49.896140 449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:49.890496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.193:42775 (local address 127.0.0.1:43090)
0923 23:14:49.891730 (+ 1234us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:49.891739 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:49.891785 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:49.892030 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:49.892030 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:49.892032 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:14:49.892077 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:49.892078 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:49.892729 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:49.892729 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:49.893705 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:49.893707 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:49.895904 (+ 2197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:49.895905 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:49.895967 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:49.895970 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 23:14:49.896028 (+ 58us) client_negotiation.cc:240] Negotiation successful
0923 23:14:49.896075 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1216,"thread_start_us":60,"threads_started":1}
I20260923 23:14:49.896190 450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:49.890659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43090 (local address 127.24.227.193:42775)
0923 23:14:49.891854 (+ 1195us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:49.891856 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:49.891908 (+ 52us) server_negotiation.cc:407] Connection header received
0923 23:14:49.891933 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:49.891933 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:49.891943 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:49.892027 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:14:49.892141 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:49.892668 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:49.893781 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:49.894344 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:49.894425 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:49.894435 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:49.896049 (+ 1614us) server_negotiation.cc:299] Negotiation successful
0923 23:14:49.896103 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1178,"thread_start_us":56,"threads_started":1}
I20260923 23:14:49.896255 451 tablet_bootstrap.cc:654] T b10e9ad7b61744f38e0f67a22cba3177 P 99b029603b8e4c9fbafedaafc91ea951: Neither blocks nor log segments found. Creating new log.
I20260923 23:14:49.896493 32479 tablet_service.cc:1467] Processing CreateTablet for tablet b10e9ad7b61744f38e0f67a22cba3177 (DEFAULT_TABLE table=test-table [id=0d46eb4193a14b77b42effbf3d9c5c79]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 23:14:49.896591 32479 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b10e9ad7b61744f38e0f67a22cba3177. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:14:49.896818 451 tablet_bootstrap.cc:492] T b10e9ad7b61744f38e0f67a22cba3177 P 99b029603b8e4c9fbafedaafc91ea951: No bootstrap required, opened a new log
I20260923 23:14:49.896885 451 ts_tablet_manager.cc:1397] T b10e9ad7b61744f38e0f67a22cba3177 P 99b029603b8e4c9fbafedaafc91ea951: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260923 23:14:49.896981 452 tablet_bootstrap.cc:654] T b10e9ad7b61744f38e0f67a22cba3177 P 6eb81312874d421c9884f64eff92eae6: Neither blocks nor log segments found. Creating new log.
I20260923 23:14:49.897024 451 raft_consensus.cc:348] T b10e9ad7b61744f38e0f67a22cba3177 P 99b029603b8e4c9fbafedaafc91ea951 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eb81312874d421c9884f64eff92eae6" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 43865 } } peers { permanent_uuid: "99b029603b8e4c9fbafedaafc91ea951" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 33365 } } peers { permanent_uuid: "85902a6195e544b7a8e8794e0b7c7ca1" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 42775 } }
I20260923 23:14:49.897091 451 raft_consensus.cc:374] T b10e9ad7b61744f38e0f67a22cba3177 P 99b029603b8e4c9fbafedaafc91ea951 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:14:49.897109 451 raft_consensus.cc:729] T b10e9ad7b61744f38e0f67a22cba3177 P 99b029603b8e4c9fbafedaafc91ea951 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99b029603b8e4c9fbafedaafc91ea951, State: Initialized, Role: FOLLOWER
I20260923 23:14:49.897152 451 consensus_queue.cc:260] T b10e9ad7b61744f38e0f67a22cba3177 P 99b029603b8e4c9fbafedaafc91ea951 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eb81312874d421c9884f64eff92eae6" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 43865 } } peers { permanent_uuid: "99b029603b8e4c9fbafedaafc91ea951" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 33365 } } peers { permanent_uuid: "85902a6195e544b7a8e8794e0b7c7ca1" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 42775 } }
I20260923 23:14:49.897298 451 ts_tablet_manager.cc:1428] T b10e9ad7b61744f38e0f67a22cba3177 P 99b029603b8e4c9fbafedaafc91ea951: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260923 23:14:49.897751 456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:49.897457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41951 (local address 127.24.227.194:60397)
0923 23:14:49.897659 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:49.897694 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:41951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":83,"threads_started":1}
I20260923 23:14:49.897850 32683 heartbeater.cc:503] Master 127.24.227.253:35161 was elected leader, sending a full tablet report...
I20260923 23:14:49.898152 457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:49.897945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41951 (local address 127.24.227.194:60157)
0923 23:14:49.898067 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:49.898104 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:41951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":85,"threads_started":1}
I20260923 23:14:49.898267 458 tablet_bootstrap.cc:492] T b10e9ad7b61744f38e0f67a22cba3177 P 85902a6195e544b7a8e8794e0b7c7ca1: Bootstrap starting.
I20260923 23:14:49.898677 458 tablet_bootstrap.cc:654] T b10e9ad7b61744f38e0f67a22cba3177 P 85902a6195e544b7a8e8794e0b7c7ca1: Neither blocks nor log segments found. Creating new log.
I20260923 23:14:49.899163 458 tablet_bootstrap.cc:492] T b10e9ad7b61744f38e0f67a22cba3177 P 85902a6195e544b7a8e8794e0b7c7ca1: No bootstrap required, opened a new log
I20260923 23:14:49.899185 452 tablet_bootstrap.cc:492] T b10e9ad7b61744f38e0f67a22cba3177 P 6eb81312874d421c9884f64eff92eae6: No bootstrap required, opened a new log
I20260923 23:14:49.899217 452 ts_tablet_manager.cc:1397] T b10e9ad7b61744f38e0f67a22cba3177 P 6eb81312874d421c9884f64eff92eae6: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260923 23:14:49.899226 458 ts_tablet_manager.cc:1397] T b10e9ad7b61744f38e0f67a22cba3177 P 85902a6195e544b7a8e8794e0b7c7ca1: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260923 23:14:49.899323 458 raft_consensus.cc:348] T b10e9ad7b61744f38e0f67a22cba3177 P 85902a6195e544b7a8e8794e0b7c7ca1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eb81312874d421c9884f64eff92eae6" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 43865 } } peers { permanent_uuid: "99b029603b8e4c9fbafedaafc91ea951" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 33365 } } peers { permanent_uuid: "85902a6195e544b7a8e8794e0b7c7ca1" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 42775 } }
I20260923 23:14:49.899346 452 raft_consensus.cc:348] T b10e9ad7b61744f38e0f67a22cba3177 P 6eb81312874d421c9884f64eff92eae6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eb81312874d421c9884f64eff92eae6" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 43865 } } peers { permanent_uuid: "99b029603b8e4c9fbafedaafc91ea951" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 33365 } } peers { permanent_uuid: "85902a6195e544b7a8e8794e0b7c7ca1" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 42775 } }
I20260923 23:14:49.899422 452 raft_consensus.cc:374] T b10e9ad7b61744f38e0f67a22cba3177 P 6eb81312874d421c9884f64eff92eae6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:14:49.899422 458 raft_consensus.cc:374] T b10e9ad7b61744f38e0f67a22cba3177 P 85902a6195e544b7a8e8794e0b7c7ca1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:14:49.899435 452 raft_consensus.cc:729] T b10e9ad7b61744f38e0f67a22cba3177 P 6eb81312874d421c9884f64eff92eae6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6eb81312874d421c9884f64eff92eae6, State: Initialized, Role: FOLLOWER
I20260923 23:14:49.899464 458 raft_consensus.cc:729] T b10e9ad7b61744f38e0f67a22cba3177 P 85902a6195e544b7a8e8794e0b7c7ca1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85902a6195e544b7a8e8794e0b7c7ca1, State: Initialized, Role: FOLLOWER
I20260923 23:14:49.899528 452 consensus_queue.cc:260] T b10e9ad7b61744f38e0f67a22cba3177 P 6eb81312874d421c9884f64eff92eae6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eb81312874d421c9884f64eff92eae6" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 43865 } } peers { permanent_uuid: "99b029603b8e4c9fbafedaafc91ea951" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 33365 } } peers { permanent_uuid: "85902a6195e544b7a8e8794e0b7c7ca1" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 42775 } }
I20260923 23:14:49.899567 458 consensus_queue.cc:260] T b10e9ad7b61744f38e0f67a22cba3177 P 85902a6195e544b7a8e8794e0b7c7ca1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eb81312874d421c9884f64eff92eae6" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 43865 } } peers { permanent_uuid: "99b029603b8e4c9fbafedaafc91ea951" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 33365 } } peers { permanent_uuid: "85902a6195e544b7a8e8794e0b7c7ca1" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 42775 } }
I20260923 23:14:49.899688 452 ts_tablet_manager.cc:1428] T b10e9ad7b61744f38e0f67a22cba3177 P 6eb81312874d421c9884f64eff92eae6: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260923 23:14:49.899717 458 ts_tablet_manager.cc:1428] T b10e9ad7b61744f38e0f67a22cba3177 P 85902a6195e544b7a8e8794e0b7c7ca1: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260923 23:14:49.900255 463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:49.900052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41951 (local address 127.24.227.193:46857)
0923 23:14:49.900182 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:49.900217 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:41951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":67,"threads_started":1}
I20260923 23:14:49.900486 462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:49.899966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41951 (local address 127.24.227.195:54051)
0923 23:14:49.900412 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:49.900444 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:41951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":83,"threads_started":1}
I20260923 23:14:49.900604 463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:49.900498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41951 (local address 127.24.227.193:59621)
0923 23:14:49.900560 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:49.900586 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:41951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260923 23:14:49.901093 32547 heartbeater.cc:503] Master 127.24.227.253:35161 was elected leader, sending a full tablet report...
I20260923 23:14:49.901620 351 heartbeater.cc:503] Master 127.24.227.253:35161 was elected leader, sending a full tablet report...
I20260923 23:14:49.901758 462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:49.901668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41951 (local address 127.24.227.195:59089)
0923 23:14:49.901714 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:49.901735 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:41951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260923 23:14:49.983065 455 raft_consensus.cc:482] T b10e9ad7b61744f38e0f67a22cba3177 P 99b029603b8e4c9fbafedaafc91ea951 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 23:14:49.983157 455 raft_consensus.cc:504] T b10e9ad7b61744f38e0f67a22cba3177 P 99b029603b8e4c9fbafedaafc91ea951 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eb81312874d421c9884f64eff92eae6" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 43865 } } peers { permanent_uuid: "99b029603b8e4c9fbafedaafc91ea951" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 33365 } } peers { permanent_uuid: "85902a6195e544b7a8e8794e0b7c7ca1" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 42775 } }
I20260923 23:14:49.983528 455 leader_election.cc:288] T b10e9ad7b61744f38e0f67a22cba3177 P 99b029603b8e4c9fbafedaafc91ea951 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6eb81312874d421c9884f64eff92eae6 (127.24.227.195:43865), 85902a6195e544b7a8e8794e0b7c7ca1 (127.24.227.193:42775)
I20260923 23:14:49.987295 447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:49.983745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.194:40947 (local address 127.24.227.195:43865)
0923 23:14:49.983803 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:49.983806 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:49.983818 (+ 12us) server_negotiation.cc:407] Connection header received
0923 23:14:49.983911 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:49.983913 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:49.983923 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:49.983982 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:14:49.984150 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:49.984890 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:49.986418 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:49.986964 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:49.987038 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:49.987047 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:49.987195 (+ 148us) server_negotiation.cc:299] Negotiation successful
0923 23:14:49.987233 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260923 23:14:49.987295 450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:49.983894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.194:54467 (local address 127.24.227.193:42775)
0923 23:14:49.983935 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:49.983937 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:49.983944 (+ 7us) server_negotiation.cc:407] Connection header received
0923 23:14:49.983968 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:49.983969 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:49.983978 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:49.984029 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:14:49.984149 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:49.984896 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:49.986418 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:49.986973 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:49.987038 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:49.987047 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:49.987195 (+ 148us) server_negotiation.cc:299] Negotiation successful
0923 23:14:49.987233 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260923 23:14:49.987321 457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:49.983685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.195:43865 (local address 127.24.227.194:40947)
0923 23:14:49.983753 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:49.983764 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:49.983813 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:49.984000 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:49.984001 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:49.984003 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:14:49.984073 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:49.984074 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:49.984974 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:49.984974 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:49.986325 (+ 1351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:49.986328 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:49.987069 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:49.987070 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:49.987120 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:49.987122 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 23:14:49.987173 (+ 51us) client_negotiation.cc:240] Negotiation successful
0923 23:14:49.987221 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47,"mutex_wait_us":71}
I20260923 23:14:49.987321 456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:49.983786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.193:42775 (local address 127.24.227.194:54467)
0923 23:14:49.983830 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:49.983838 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:49.983912 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:49.984027 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:49.984028 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:49.984029 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:14:49.984072 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:49.984074 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:49.984972 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:49.984973 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:49.986325 (+ 1352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:49.986328 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:49.987049 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:49.987051 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:49.987112 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:49.987117 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 23:14:49.987173 (+ 56us) client_negotiation.cc:240] Negotiation successful
0923 23:14:49.987221 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33,"mutex_wait_us":83}
I20260923 23:14:49.987633 32499 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b10e9ad7b61744f38e0f67a22cba3177" candidate_uuid: "99b029603b8e4c9fbafedaafc91ea951" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85902a6195e544b7a8e8794e0b7c7ca1" is_pre_election: true
I20260923 23:14:49.987648 303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b10e9ad7b61744f38e0f67a22cba3177" candidate_uuid: "99b029603b8e4c9fbafedaafc91ea951" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6eb81312874d421c9884f64eff92eae6" is_pre_election: true
I20260923 23:14:49.987733 303 raft_consensus.cc:2454] T b10e9ad7b61744f38e0f67a22cba3177 P 6eb81312874d421c9884f64eff92eae6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99b029603b8e4c9fbafedaafc91ea951 in term 0.
I20260923 23:14:49.987773 32499 raft_consensus.cc:2454] T b10e9ad7b61744f38e0f67a22cba3177 P 85902a6195e544b7a8e8794e0b7c7ca1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99b029603b8e4c9fbafedaafc91ea951 in term 0.
I20260923 23:14:49.987932 32568 leader_election.cc:302] T b10e9ad7b61744f38e0f67a22cba3177 P 99b029603b8e4c9fbafedaafc91ea951 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6eb81312874d421c9884f64eff92eae6, 99b029603b8e4c9fbafedaafc91ea951; no voters:
I20260923 23:14:49.988052 455 raft_consensus.cc:2781] T b10e9ad7b61744f38e0f67a22cba3177 P 99b029603b8e4c9fbafedaafc91ea951 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 23:14:49.988089 455 raft_consensus.cc:482] T b10e9ad7b61744f38e0f67a22cba3177 P 99b029603b8e4c9fbafedaafc91ea951 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 23:14:49.988117 455 raft_consensus.cc:3037] T b10e9ad7b61744f38e0f67a22cba3177 P 99b029603b8e4c9fbafedaafc91ea951 [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:14:49.988757 455 raft_consensus.cc:504] T b10e9ad7b61744f38e0f67a22cba3177 P 99b029603b8e4c9fbafedaafc91ea951 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eb81312874d421c9884f64eff92eae6" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 43865 } } peers { permanent_uuid: "99b029603b8e4c9fbafedaafc91ea951" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 33365 } } peers { permanent_uuid: "85902a6195e544b7a8e8794e0b7c7ca1" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 42775 } }
I20260923 23:14:49.988871 455 leader_election.cc:288] T b10e9ad7b61744f38e0f67a22cba3177 P 99b029603b8e4c9fbafedaafc91ea951 [CANDIDATE]: Term 1 election: Requested vote from peers 6eb81312874d421c9884f64eff92eae6 (127.24.227.195:43865), 85902a6195e544b7a8e8794e0b7c7ca1 (127.24.227.193:42775)
I20260923 23:14:49.989049 32499 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b10e9ad7b61744f38e0f67a22cba3177" candidate_uuid: "99b029603b8e4c9fbafedaafc91ea951" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85902a6195e544b7a8e8794e0b7c7ca1"
I20260923 23:14:49.989118 32499 raft_consensus.cc:3037] T b10e9ad7b61744f38e0f67a22cba3177 P 85902a6195e544b7a8e8794e0b7c7ca1 [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:14:49.989162 303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b10e9ad7b61744f38e0f67a22cba3177" candidate_uuid: "99b029603b8e4c9fbafedaafc91ea951" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6eb81312874d421c9884f64eff92eae6"
I20260923 23:14:49.989243 303 raft_consensus.cc:3037] T b10e9ad7b61744f38e0f67a22cba3177 P 6eb81312874d421c9884f64eff92eae6 [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:14:49.989777 32499 raft_consensus.cc:2454] T b10e9ad7b61744f38e0f67a22cba3177 P 85902a6195e544b7a8e8794e0b7c7ca1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99b029603b8e4c9fbafedaafc91ea951 in term 1.
I20260923 23:14:49.989822 303 raft_consensus.cc:2454] T b10e9ad7b61744f38e0f67a22cba3177 P 6eb81312874d421c9884f64eff92eae6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99b029603b8e4c9fbafedaafc91ea951 in term 1.
I20260923 23:14:49.989952 32568 leader_election.cc:302] T b10e9ad7b61744f38e0f67a22cba3177 P 99b029603b8e4c9fbafedaafc91ea951 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85902a6195e544b7a8e8794e0b7c7ca1, 99b029603b8e4c9fbafedaafc91ea951; no voters:
I20260923 23:14:49.990046 455 raft_consensus.cc:2781] T b10e9ad7b61744f38e0f67a22cba3177 P 99b029603b8e4c9fbafedaafc91ea951 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 23:14:49.990197 455 raft_consensus.cc:686] T b10e9ad7b61744f38e0f67a22cba3177 P 99b029603b8e4c9fbafedaafc91ea951 [term 1 LEADER]: Becoming Leader. State: Replica: 99b029603b8e4c9fbafedaafc91ea951, State: Running, Role: LEADER
I20260923 23:14:49.990260 455 consensus_queue.cc:237] T b10e9ad7b61744f38e0f67a22cba3177 P 99b029603b8e4c9fbafedaafc91ea951 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eb81312874d421c9884f64eff92eae6" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 43865 } } peers { permanent_uuid: "99b029603b8e4c9fbafedaafc91ea951" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 33365 } } peers { permanent_uuid: "85902a6195e544b7a8e8794e0b7c7ca1" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 42775 } }
I20260923 23:14:49.990533 456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:49.990433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41951 (local address 127.24.227.194:38323)
0923 23:14:49.990479 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:49.990511 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:41951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
I20260923 23:14:49.991081 32177 catalog_manager.cc:5495] T b10e9ad7b61744f38e0f67a22cba3177 P 99b029603b8e4c9fbafedaafc91ea951 reported cstate change: term changed from 0 to 1, leader changed from <none> to 99b029603b8e4c9fbafedaafc91ea951 (127.24.227.194). New cstate: current_term: 1 leader_uuid: "99b029603b8e4c9fbafedaafc91ea951" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eb81312874d421c9884f64eff92eae6" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 43865 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "99b029603b8e4c9fbafedaafc91ea951" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 33365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85902a6195e544b7a8e8794e0b7c7ca1" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 42775 } health_report { overall_health: UNKNOWN } } }
I20260923 23:14:50.008586 448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:50.006672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54720 (local address 127.24.227.194:33365)
0923 23:14:50.006729 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:50.006730 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:50.006737 (+ 7us) server_negotiation.cc:407] Connection header received
0923 23:14:50.006762 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:50.006762 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:50.006776 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:50.006831 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 23:14:50.006942 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:50.007357 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:50.008065 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:50.008233 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:50.008279 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:50.008394 (+ 115us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 23:14:50.008477 (+ 83us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 23:14:50.008518 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:50.008534 (+ 16us) server_negotiation.cc:299] Negotiation successful
0923 23:14:50.008548 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260923 23:14:50.009466 303 raft_consensus.cc:1261] T b10e9ad7b61744f38e0f67a22cba3177 P 6eb81312874d421c9884f64eff92eae6 [term 1 FOLLOWER]: Refusing update from remote peer 99b029603b8e4c9fbafedaafc91ea951: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 23:14:50.009575 32499 raft_consensus.cc:1261] T b10e9ad7b61744f38e0f67a22cba3177 P 85902a6195e544b7a8e8794e0b7c7ca1 [term 1 FOLLOWER]: Refusing update from remote peer 99b029603b8e4c9fbafedaafc91ea951: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 23:14:50.009707 455 consensus_queue.cc:1035] T b10e9ad7b61744f38e0f67a22cba3177 P 99b029603b8e4c9fbafedaafc91ea951 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6eb81312874d421c9884f64eff92eae6" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 43865 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 23:14:50.009817 464 consensus_queue.cc:1035] T b10e9ad7b61744f38e0f67a22cba3177 P 99b029603b8e4c9fbafedaafc91ea951 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85902a6195e544b7a8e8794e0b7c7ca1" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 42775 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 23:14:50.010492 462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:50.010320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41951 (local address 127.24.227.195:40991)
0923 23:14:50.010432 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:50.010463 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:41951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260923 23:14:50.010752 463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:50.010386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41951 (local address 127.24.227.193:42909)
0923 23:14:50.010689 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:50.010720 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:41951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":286}
I20260923 23:14:50.011438 456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:50.011326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41951 (local address 127.24.227.194:54967)
0923 23:14:50.011388 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:50.011410 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:41951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260923 23:14:50.011835 462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:50.011757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41951 (local address 127.24.227.195:48327)
0923 23:14:50.011790 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:50.011812 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:41951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260923 23:14:50.013609 456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:50.013531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41951 (local address 127.24.227.194:55963)
0923 23:14:50.013560 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:50.013587 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:41951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
I20260923 23:14:50.013784 463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:50.011890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41951 (local address 127.24.227.193:39807)
0923 23:14:50.013584 (+ 1694us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:50.013634 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:41951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1680}
I20260923 23:14:50.014441 32177 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50662:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 23:14:50.014521 32177 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50662:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 23:14:50.015795 32177 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bd675df8e6274c688bb00c0f39e018c1: Sending DeleteTablet for 3 replicas of tablet b10e9ad7b61744f38e0f67a22cba3177
I20260923 23:14:50.016206 32615 tablet_service.cc:1514] Processing DeleteTablet for tablet b10e9ad7b61744f38e0f67a22cba3177 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 23:14:50 UTC) from {username='oryx', principal='oryx/127.24.227.253@KRBTEST.COM'} at 127.0.0.1:54718
I20260923 23:14:50.016230 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 32416
I20260923 23:14:50.016311 32751 tablet_service.cc:1514] Processing DeleteTablet for tablet b10e9ad7b61744f38e0f67a22cba3177 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 23:14:50 UTC) from {username='oryx', principal='oryx/127.24.227.253@KRBTEST.COM'} at 127.0.0.1:52208
I20260923 23:14:50.016441 32479 tablet_service.cc:1514] Processing DeleteTablet for tablet b10e9ad7b61744f38e0f67a22cba3177 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 23:14:50 UTC) from {username='oryx', principal='oryx/127.24.227.253@KRBTEST.COM'} at 127.0.0.1:43090
I20260923 23:14:50.019963 479 tablet_replica.cc:330] T b10e9ad7b61744f38e0f67a22cba3177 P 6eb81312874d421c9884f64eff92eae6: stopping tablet replica
I20260923 23:14:50.020031 479 raft_consensus.cc:2229] T b10e9ad7b61744f38e0f67a22cba3177 P 6eb81312874d421c9884f64eff92eae6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 23:14:50.020078 479 raft_consensus.cc:2258] T b10e9ad7b61744f38e0f67a22cba3177 P 6eb81312874d421c9884f64eff92eae6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 23:14:50.020192 478 tablet_replica.cc:330] T b10e9ad7b61744f38e0f67a22cba3177 P 99b029603b8e4c9fbafedaafc91ea951: stopping tablet replica
I20260923 23:14:50.020258 478 raft_consensus.cc:2229] T b10e9ad7b61744f38e0f67a22cba3177 P 99b029603b8e4c9fbafedaafc91ea951 [term 1 LEADER]: Raft consensus shutting down.
I20260923 23:14:50.020332 478 raft_consensus.cc:2258] T b10e9ad7b61744f38e0f67a22cba3177 P 99b029603b8e4c9fbafedaafc91ea951 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 23:14:50.020378 479 ts_tablet_manager.cc:1905] T b10e9ad7b61744f38e0f67a22cba3177 P 6eb81312874d421c9884f64eff92eae6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 23:14:50.020571 478 ts_tablet_manager.cc:1905] T b10e9ad7b61744f38e0f67a22cba3177 P 99b029603b8e4c9fbafedaafc91ea951: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 23:14:50.021934 478 ts_tablet_manager.cc:1918] T b10e9ad7b61744f38e0f67a22cba3177 P 99b029603b8e4c9fbafedaafc91ea951: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 23:14:50.022014 478 log.cc:1196] T b10e9ad7b61744f38e0f67a22cba3177 P 99b029603b8e4c9fbafedaafc91ea951: Deleting WAL directory at /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/ts-1/wal/wals/b10e9ad7b61744f38e0f67a22cba3177
I20260923 23:14:50.022224 478 ts_tablet_manager.cc:1939] T b10e9ad7b61744f38e0f67a22cba3177 P 99b029603b8e4c9fbafedaafc91ea951: Deleting consensus metadata
I20260923 23:14:50.022693 32161 catalog_manager.cc:4841] TS 99b029603b8e4c9fbafedaafc91ea951 (127.24.227.194:33365): tablet b10e9ad7b61744f38e0f67a22cba3177 (table test-table [id=0d46eb4193a14b77b42effbf3d9c5c79]) successfully deleted
I20260923 23:14:50.023972 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 32552
I20260923 23:14:50.024086 449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:50.023962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.193:42775 (local address 127.0.0.1:43100)
0923 23:14:50.024019 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:50.024052 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.193:42775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260923 23:14:50.024654 32161 catalog_manager.cc:4584] TS 85902a6195e544b7a8e8794e0b7c7ca1 (127.24.227.193:42775): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b10e9ad7b61744f38e0f67a22cba3177: Network error: Client connection negotiation failed: client connection to 127.24.227.193:42775: connect: Connection refused (error 111)
I20260923 23:14:50.024847 479 ts_tablet_manager.cc:1918] T b10e9ad7b61744f38e0f67a22cba3177 P 6eb81312874d421c9884f64eff92eae6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 23:14:50.024930 479 log.cc:1196] T b10e9ad7b61744f38e0f67a22cba3177 P 6eb81312874d421c9884f64eff92eae6: Deleting WAL directory at /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790205259943518-25487-0/minicluster-data/ts-2/wal/wals/b10e9ad7b61744f38e0f67a22cba3177
I20260923 23:14:50.025143 479 ts_tablet_manager.cc:1939] T b10e9ad7b61744f38e0f67a22cba3177 P 6eb81312874d421c9884f64eff92eae6: Deleting consensus metadata
I20260923 23:14:50.025454 32161 catalog_manager.cc:4841] TS 6eb81312874d421c9884f64eff92eae6 (127.24.227.195:43865): tablet b10e9ad7b61744f38e0f67a22cba3177 (table test-table [id=0d46eb4193a14b77b42effbf3d9c5c79]) successfully deleted
I20260923 23:14:50.029618 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 32688
I20260923 23:14:50.033856 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 32037
I20260923 23:14:50.038434 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 32231
2026-09-23T23:14:50Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (6677 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 23:14:50 dist-test-slave-754n krb5kdc[482](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 23:14:50 dist-test-slave-754n krb5kdc[482](info): set up 2 sockets
Sep 23 23:14:50 dist-test-slave-754n krb5kdc[482](info): commencing operation
krb5kdc: starting...
W20260923 23:14:51.032011 25487 mini_kdc.cc:121] Time spent starting KDC: real 0.973s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/krb5kdc/test-user.keytab.
Sep 23 23:14:51 dist-test-slave-754n krb5kdc[482](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205291, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T23:14:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T23:14:51Z Disabled control of system clock
WARNING: no policy specified for oryx/127.24.227.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.227.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.227.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/krb5kdc/oryx_127.24.227.254.keytab.
Entry for principal oryx/127.24.227.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/krb5kdc/oryx_127.24.227.254.keytab.
I20260923 23:14:51.136245 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.254:34607
--webserver_interface=127.24.227.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/krb5kdc/oryx_127.24.227.254.keytab
--principal=oryx/127.24.227.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:35707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.227.254:34607
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:14:51.211344 495 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:14:51.211534 495 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:14:51.211567 495 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:14:51.212612 495 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 23:14:51.212671 495 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260923 23:14:51.212690 495 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:14:51.212709 495 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:14:51.212738 495 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 23:14:51.212759 495 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 23:14:51.214133 495 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:35707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.227.254:34607
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/krb5kdc/oryx_127.24.227.254.keytab
--principal=oryx/127.24.227.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.254:34607
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.227.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.495
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:14:51.214377 495 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:14:51.214598 495 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:14:51.219062 501 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:51.219070 500 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:14:51.219231 495 server_base.cc:1034] running on GCE node
W20260923 23:14:51.219069 503 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:14:51.219524 495 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:14:51.219833 495 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:14:51.220999 495 hybrid_clock.cc:648] HybridClock initialized: now 1790205291220984 us; error 31 us; skew 500 ppm
Sep 23 23:14:51 dist-test-slave-754n krb5kdc[482](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205291, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.227.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:14:51.228549 495 init.cc:377] Logged in from keytab as oryx/127.24.227.254@KRBTEST.COM (short username oryx)
I20260923 23:14:51.229041 495 webserver.cc:460] Webserver started at http://127.24.227.254:41291/ using document root <none> and password file <none>
I20260923 23:14:51.229270 495 fs_manager.cc:359] Metadata directory not provided
I20260923 23:14:51.229314 495 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:14:51.229451 495 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:14:51.230403 495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/minicluster-data/master-0/data/instance:
uuid: "607f68c4a2fb4f858b83bbc07edd2bdb"
format_stamp: "Formatted at 2026-09-23 23:14:51 on dist-test-slave-754n"
server_key: "bdea88b7360e053d233e188e83b1af5f"
server_key_iv: "e011f9102c3e7a97c427099bcb119c2e"
server_key_version: "encryptionkey@0"
I20260923 23:14:51.230749 495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/minicluster-data/master-0/wal/instance:
uuid: "607f68c4a2fb4f858b83bbc07edd2bdb"
format_stamp: "Formatted at 2026-09-23 23:14:51 on dist-test-slave-754n"
server_key: "bdea88b7360e053d233e188e83b1af5f"
server_key_iv: "e011f9102c3e7a97c427099bcb119c2e"
server_key_version: "encryptionkey@0"
I20260923 23:14:51.232015 495 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260923 23:14:51.232717 510 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:51.232896 495 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 23:14:51.232991 495 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/minicluster-data/master-0/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/minicluster-data/master-0/wal
uuid: "607f68c4a2fb4f858b83bbc07edd2bdb"
format_stamp: "Formatted at 2026-09-23 23:14:51 on dist-test-slave-754n"
server_key: "bdea88b7360e053d233e188e83b1af5f"
server_key_iv: "e011f9102c3e7a97c427099bcb119c2e"
server_key_version: "encryptionkey@0"
I20260923 23:14:51.233075 495 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:14:51.264635 495 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:14:51.266278 495 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:14:51.266429 495 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:14:51.270793 495 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.254:34607
I20260923 23:14:51.270825 572 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.254:34607 every 8 connection(s)
I20260923 23:14:51.271122 495 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
I20260923 23:14:51.271703 573 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:14:51.273738 573 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 607f68c4a2fb4f858b83bbc07edd2bdb: Bootstrap starting.
I20260923 23:14:51.274287 573 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 607f68c4a2fb4f858b83bbc07edd2bdb: Neither blocks nor log segments found. Creating new log.
I20260923 23:14:51.274569 573 log.cc:824] T 00000000000000000000000000000000 P 607f68c4a2fb4f858b83bbc07edd2bdb: Log is configured to *not* fsync() on all Append() calls
I20260923 23:14:51.275206 573 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 607f68c4a2fb4f858b83bbc07edd2bdb: No bootstrap required, opened a new log
I20260923 23:14:51.277185 573 raft_consensus.cc:348] T 00000000000000000000000000000000 P 607f68c4a2fb4f858b83bbc07edd2bdb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "607f68c4a2fb4f858b83bbc07edd2bdb" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 34607 } }
I20260923 23:14:51.277307 573 raft_consensus.cc:374] T 00000000000000000000000000000000 P 607f68c4a2fb4f858b83bbc07edd2bdb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:14:51.277328 573 raft_consensus.cc:729] T 00000000000000000000000000000000 P 607f68c4a2fb4f858b83bbc07edd2bdb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 607f68c4a2fb4f858b83bbc07edd2bdb, State: Initialized, Role: FOLLOWER
I20260923 23:14:51.277491 573 consensus_queue.cc:260] T 00000000000000000000000000000000 P 607f68c4a2fb4f858b83bbc07edd2bdb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "607f68c4a2fb4f858b83bbc07edd2bdb" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 34607 } }
I20260923 23:14:51.277549 573 raft_consensus.cc:388] T 00000000000000000000000000000000 P 607f68c4a2fb4f858b83bbc07edd2bdb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 23:14:51.277590 573 raft_consensus.cc:482] T 00000000000000000000000000000000 P 607f68c4a2fb4f858b83bbc07edd2bdb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 23:14:51.277644 573 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 607f68c4a2fb4f858b83bbc07edd2bdb [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:14:51.278120 573 raft_consensus.cc:504] T 00000000000000000000000000000000 P 607f68c4a2fb4f858b83bbc07edd2bdb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "607f68c4a2fb4f858b83bbc07edd2bdb" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 34607 } }
I20260923 23:14:51.278226 573 leader_election.cc:302] T 00000000000000000000000000000000 P 607f68c4a2fb4f858b83bbc07edd2bdb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 607f68c4a2fb4f858b83bbc07edd2bdb; no voters:
I20260923 23:14:51.278401 573 leader_election.cc:288] T 00000000000000000000000000000000 P 607f68c4a2fb4f858b83bbc07edd2bdb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 23:14:51.278440 576 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 607f68c4a2fb4f858b83bbc07edd2bdb [term 1 FOLLOWER]: Leader election won for term 1
I20260923 23:14:51.278590 576 raft_consensus.cc:686] T 00000000000000000000000000000000 P 607f68c4a2fb4f858b83bbc07edd2bdb [term 1 LEADER]: Becoming Leader. State: Replica: 607f68c4a2fb4f858b83bbc07edd2bdb, State: Running, Role: LEADER
I20260923 23:14:51.278650 573 sys_catalog.cc:564] T 00000000000000000000000000000000 P 607f68c4a2fb4f858b83bbc07edd2bdb [sys.catalog]: configured and running, proceeding with master startup.
I20260923 23:14:51.278674 576 consensus_queue.cc:237] T 00000000000000000000000000000000 P 607f68c4a2fb4f858b83bbc07edd2bdb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "607f68c4a2fb4f858b83bbc07edd2bdb" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 34607 } }
I20260923 23:14:51.279058 577 sys_catalog.cc:455] T 00000000000000000000000000000000 P 607f68c4a2fb4f858b83bbc07edd2bdb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "607f68c4a2fb4f858b83bbc07edd2bdb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "607f68c4a2fb4f858b83bbc07edd2bdb" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 34607 } } }
I20260923 23:14:51.279173 577 sys_catalog.cc:458] T 00000000000000000000000000000000 P 607f68c4a2fb4f858b83bbc07edd2bdb [sys.catalog]: This master's current role is: LEADER
I20260923 23:14:51.279047 578 sys_catalog.cc:455] T 00000000000000000000000000000000 P 607f68c4a2fb4f858b83bbc07edd2bdb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 607f68c4a2fb4f858b83bbc07edd2bdb. Latest consensus state: current_term: 1 leader_uuid: "607f68c4a2fb4f858b83bbc07edd2bdb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "607f68c4a2fb4f858b83bbc07edd2bdb" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 34607 } } }
I20260923 23:14:51.279354 585 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 23:14:51.279363 578 sys_catalog.cc:458] T 00000000000000000000000000000000 P 607f68c4a2fb4f858b83bbc07edd2bdb [sys.catalog]: This master's current role is: LEADER
I20260923 23:14:51.279824 585 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 23:14:51.280658 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 495
I20260923 23:14:51.280768 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/minicluster-data/master-0/wal/instance
I20260923 23:14:51.280997 25487 external_mini_cluster.cc:1428] Setting key 97c0a29d1c242f17091432a4a99b8575
I20260923 23:14:51.281484 585 catalog_manager.cc:1347] Generated new cluster ID: 8f8de136aa864255bf232e9e55a6529d
I20260923 23:14:51.281538 585 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 23:14:51 dist-test-slave-754n krb5kdc[482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205291, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM
I20260923 23:14:51.292840 596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:51.281450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58816 (local address 127.24.227.254:34607)
0923 23:14:51.284043 (+ 2593us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:51.284049 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:51.284225 (+ 176us) server_negotiation.cc:407] Connection header received
0923 23:14:51.284356 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:51.284361 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:51.284399 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:51.284514 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:51.285132 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:51.286349 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:51.287094 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:51.287543 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:51.287650 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:51.289914 (+ 2264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:51.289918 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:51.289918 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:51.289935 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:51.291395 (+ 1460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:51.291743 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:51.291745 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:51.291746 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:51.291824 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:51.292030 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:51.292031 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:51.292031 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:51.292261 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:51.292395 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:51.292513 (+ 118us) server_negotiation.cc:299] Negotiation successful
0923 23:14:51.292652 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2563,"thread_start_us":75,"threads_started":1}
I20260923 23:14:51.302251 585 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 23:14:51.302680 585 catalog_manager.cc:1504] Loading token signing keys...
I20260923 23:14:51.307283 585 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 607f68c4a2fb4f858b83bbc07edd2bdb: Generated new TSK 0
I20260923 23:14:51.307456 585 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.24.227.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.227.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.227.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/krb5kdc/oryx_127.24.227.193.keytab.
Entry for principal oryx/127.24.227.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/krb5kdc/oryx_127.24.227.193.keytab.
I20260923 23:14:51.334689 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.227.193:0
--local_ip_for_outbound_sockets=127.24.227.193
--webserver_interface=127.24.227.193
--webserver_port=0
--tserver_master_addrs=127.24.227.254:34607
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/krb5kdc/oryx_127.24.227.193.keytab
--principal=oryx/127.24.227.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:35707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:14:51.410934 599 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:14:51.411104 599 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:14:51.411124 599 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:14:51.412101 599 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:14:51.412129 599 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:14:51.412147 599 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260923 23:14:51.412163 599 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.227.193
I20260923 23:14:51.413496 599 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:35707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/krb5kdc/oryx_127.24.227.193.keytab
--principal=oryx/127.24.227.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.227.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.227.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.227.254:34607
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.599
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.227.193
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:14:51.413694 599 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:14:51.413842 599 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:14:51.418407 607 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:51.418422 605 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:51.418413 604 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:14:51.418524 599 server_base.cc:1034] running on GCE node
I20260923 23:14:51.418795 599 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:14:51.418999 599 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:14:51.420123 599 hybrid_clock.cc:648] HybridClock initialized: now 1790205291420113 us; error 25 us; skew 500 ppm
Sep 23 23:14:51 dist-test-slave-754n krb5kdc[482](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205291, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.227.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:14:51.422400 599 init.cc:377] Logged in from keytab as oryx/127.24.227.193@KRBTEST.COM (short username oryx)
I20260923 23:14:51.422839 599 webserver.cc:460] Webserver started at http://127.24.227.193:41573/ using document root <none> and password file <none>
I20260923 23:14:51.423034 599 fs_manager.cc:359] Metadata directory not provided
I20260923 23:14:51.423074 599 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:14:51.423192 599 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:14:51.424201 599 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/minicluster-data/ts-0/data/instance:
uuid: "2cd0dd070b8643bf966ede03f18a5de4"
format_stamp: "Formatted at 2026-09-23 23:14:51 on dist-test-slave-754n"
server_key: "8e9e5392154cdd05025f4fc67bb08978"
server_key_iv: "134f535b09a051f9c94bd5fd84e27356"
server_key_version: "encryptionkey@0"
I20260923 23:14:51.424535 599 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/minicluster-data/ts-0/wal/instance:
uuid: "2cd0dd070b8643bf966ede03f18a5de4"
format_stamp: "Formatted at 2026-09-23 23:14:51 on dist-test-slave-754n"
server_key: "8e9e5392154cdd05025f4fc67bb08978"
server_key_iv: "134f535b09a051f9c94bd5fd84e27356"
server_key_version: "encryptionkey@0"
I20260923 23:14:51.425768 599 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 23:14:51.426409 614 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:51.426589 599 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 23:14:51.426680 599 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/minicluster-data/ts-0/wal
uuid: "2cd0dd070b8643bf966ede03f18a5de4"
format_stamp: "Formatted at 2026-09-23 23:14:51 on dist-test-slave-754n"
server_key: "8e9e5392154cdd05025f4fc67bb08978"
server_key_iv: "134f535b09a051f9c94bd5fd84e27356"
server_key_version: "encryptionkey@0"
I20260923 23:14:51.426750 599 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:14:51.443097 599 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:14:51.444810 599 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:14:51.444947 599 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:14:51.445473 599 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 23:14:51.445523 599 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:51.445555 599 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 23:14:51.445598 599 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 23 23:14:51 dist-test-slave-754n krb5kdc[482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205291, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.227.193@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM
I20260923 23:14:51.452491 599 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.193:43785
I20260923 23:14:51.452569 729 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.193:43785 every 8 connection(s)
I20260923 23:14:51.452888 599 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/minicluster-data/ts-0/data/info.pb
I20260923 23:14:51.454563 596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:51.445788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.193:36237 (local address 127.24.227.254:34607)
0923 23:14:51.445838 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:51.445841 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:51.446065 (+ 224us) server_negotiation.cc:407] Connection header received
0923 23:14:51.446465 (+ 400us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:51.446466 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:51.446479 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:51.446555 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:51.447219 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:51.447740 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:51.448807 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:51.449018 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:51.449100 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:51.451997 (+ 2897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:51.451999 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:51.451999 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:51.452006 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:51.453561 (+ 1555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:51.453881 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:51.453881 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:51.453882 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:51.453928 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:51.454111 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:51.454112 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:51.454112 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:51.454209 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:51.454254 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:51.454434 (+ 180us) server_negotiation.cc:299] Negotiation successful
0923 23:14:51.454488 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260923 23:14:51.454766 623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:51.445791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:34607 (local address 127.24.227.193:36237)
0923 23:14:51.445923 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:51.445955 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:51.446375 (+ 420us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:51.446609 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:51.446612 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:51.446943 (+ 331us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:14:51.447120 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:51.447125 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:51.447831 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:51.447833 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:51.448721 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:51.448723 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:51.449111 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:51.449112 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:51.449235 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:51.449718 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:14:51.449729 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:14:51.451908 (+ 2179us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:14:51.453626 (+ 1718us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:51.453630 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:51.453631 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:51.453817 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:51.453980 (+ 163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:51.453980 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:51.453981 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:51.454062 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:51.454258 (+ 196us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:14:51.454261 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:14:51.454354 (+ 93us) client_negotiation.cc:769] Sending connection context
0923 23:14:51.454423 (+ 69us) client_negotiation.cc:240] Negotiation successful
0923 23:14:51.454521 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":72,"threads_started":1}
I20260923 23:14:51.455250 730 heartbeater.cc:347] Connected to a master server at 127.24.227.254:34607
I20260923 23:14:51.455340 730 heartbeater.cc:464] Registering TS with master...
I20260923 23:14:51.455492 730 heartbeater.cc:511] Master 127.24.227.254:34607 requested a full tablet report, sending...
I20260923 23:14:51.456025 527 ts_manager.cc:194] Registered new tserver with Master: 2cd0dd070b8643bf966ede03f18a5de4 (127.24.227.193:43785)
I20260923 23:14:51.456629 527 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.227.193@KRBTEST.COM'} at 127.24.227.193:36237
I20260923 23:14:51.458859 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 599
I20260923 23:14:51.458974 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/minicluster-data/ts-0/wal/instance
I20260923 23:14:51.459179 25487 external_mini_cluster.cc:1428] Setting key a4b479b83f66f72f287565ec519aa352
I20260923 23:14:51.463330 596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:51.457036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.193:38267 (local address 127.24.227.254:34607)
0923 23:14:51.457481 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:51.457483 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:51.457490 (+ 7us) server_negotiation.cc:407] Connection header received
0923 23:14:51.457506 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:51.457507 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:51.457514 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:51.457589 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:51.457895 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:51.458360 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:51.459362 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:51.459626 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:51.459704 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:51.460680 (+ 976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:51.460682 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:51.460682 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:51.460690 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:51.462276 (+ 1586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:51.462580 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:51.462581 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:51.462581 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:51.462631 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:51.462848 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:51.462850 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:51.462850 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:51.462955 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:51.463024 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:51.463179 (+ 155us) server_negotiation.cc:299] Negotiation successful
0923 23:14:51.463250 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":426}
I20260923 23:14:51.463340 736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:51.456966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:34607 (local address 127.24.227.193:38267)
0923 23:14:51.457086 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:51.457097 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:51.457156 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:51.457596 (+ 440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:51.457596 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:51.457772 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:14:51.457817 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:51.457818 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:51.458459 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:51.458460 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:51.459260 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:51.459262 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:51.459712 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:51.459713 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:51.459777 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:51.460192 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:14:51.460197 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:14:51.460573 (+ 376us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:14:51.462350 (+ 1777us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:51.462351 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:51.462352 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:51.462510 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:51.462710 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:51.462711 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:51.462712 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:51.462767 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:51.463029 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:14:51.463030 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:14:51.463070 (+ 40us) client_negotiation.cc:769] Sending connection context
0923 23:14:51.463156 (+ 86us) client_negotiation.cc:240] Negotiation successful
0923 23:14:51.463250 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":69,"threads_started":1}
WARNING: no policy specified for oryx/127.24.227.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.227.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.227.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/krb5kdc/oryx_127.24.227.194.keytab.
Entry for principal oryx/127.24.227.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/krb5kdc/oryx_127.24.227.194.keytab.
I20260923 23:14:51.484009 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.227.194:0
--local_ip_for_outbound_sockets=127.24.227.194
--webserver_interface=127.24.227.194
--webserver_port=0
--tserver_master_addrs=127.24.227.254:34607
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/krb5kdc/oryx_127.24.227.194.keytab
--principal=oryx/127.24.227.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:35707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:14:51.562928 739 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:14:51.563104 739 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:14:51.563125 739 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:14:51.564373 739 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:14:51.564414 739 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:14:51.564433 739 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260923 23:14:51.564486 739 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.227.194
I20260923 23:14:51.565923 739 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:35707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/krb5kdc/oryx_127.24.227.194.keytab
--principal=oryx/127.24.227.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.227.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.227.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.227.254:34607
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.739
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.227.194
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:14:51.566082 739 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:14:51.566326 739 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:14:51.571097 747 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:51.571102 744 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:51.571164 745 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:14:51.571118 739 server_base.cc:1034] running on GCE node
I20260923 23:14:51.571528 739 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:14:51.571734 739 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:14:51.572887 739 hybrid_clock.cc:648] HybridClock initialized: now 1790205291572861 us; error 37 us; skew 500 ppm
Sep 23 23:14:51 dist-test-slave-754n krb5kdc[482](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205291, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.227.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:14:51.575188 739 init.cc:377] Logged in from keytab as oryx/127.24.227.194@KRBTEST.COM (short username oryx)
I20260923 23:14:51.575608 739 webserver.cc:460] Webserver started at http://127.24.227.194:38567/ using document root <none> and password file <none>
I20260923 23:14:51.575814 739 fs_manager.cc:359] Metadata directory not provided
I20260923 23:14:51.575896 739 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:14:51.576056 739 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:14:51.577011 739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/minicluster-data/ts-1/data/instance:
uuid: "9d239c22857640e7b2ff0c7008fbfaca"
format_stamp: "Formatted at 2026-09-23 23:14:51 on dist-test-slave-754n"
server_key: "b16b08d20768bb06cc11ae2d6e34459a"
server_key_iv: "7dde1c5de47d52c64b60c2b3e5b5e672"
server_key_version: "encryptionkey@0"
I20260923 23:14:51.577373 739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/minicluster-data/ts-1/wal/instance:
uuid: "9d239c22857640e7b2ff0c7008fbfaca"
format_stamp: "Formatted at 2026-09-23 23:14:51 on dist-test-slave-754n"
server_key: "b16b08d20768bb06cc11ae2d6e34459a"
server_key_iv: "7dde1c5de47d52c64b60c2b3e5b5e672"
server_key_version: "encryptionkey@0"
I20260923 23:14:51.578691 739 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 23:14:51.579499 754 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:51.579665 739 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 23:14:51.579768 739 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/minicluster-data/ts-1/wal
uuid: "9d239c22857640e7b2ff0c7008fbfaca"
format_stamp: "Formatted at 2026-09-23 23:14:51 on dist-test-slave-754n"
server_key: "b16b08d20768bb06cc11ae2d6e34459a"
server_key_iv: "7dde1c5de47d52c64b60c2b3e5b5e672"
server_key_version: "encryptionkey@0"
I20260923 23:14:51.579838 739 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:14:51.604166 739 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:14:51.605799 739 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:14:51.605975 739 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:14:51.606521 739 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 23:14:51.606561 739 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:51.606601 739 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 23:14:51.606629 739 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 23 23:14:51 dist-test-slave-754n krb5kdc[482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205291, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.227.194@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM
I20260923 23:14:51.613619 739 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.194:38277
I20260923 23:14:51.613667 869 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.194:38277 every 8 connection(s)
I20260923 23:14:51.613961 739 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/minicluster-data/ts-1/data/info.pb
I20260923 23:14:51.616657 596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:51.606901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.194:36263 (local address 127.24.227.254:34607)
0923 23:14:51.606939 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:51.606941 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:51.607126 (+ 185us) server_negotiation.cc:407] Connection header received
0923 23:14:51.607575 (+ 449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:51.607576 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:51.607590 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:51.607665 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:51.608384 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:51.609241 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:51.610368 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:51.610568 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:51.610653 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:51.613938 (+ 3285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:51.613939 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:51.613939 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:51.613947 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:51.615508 (+ 1561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:51.615859 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:51.615860 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:51.615860 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:51.615962 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:51.616184 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:51.616184 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:51.616185 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:51.616274 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:51.616320 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:51.616521 (+ 201us) server_negotiation.cc:299] Negotiation successful
0923 23:14:51.616576 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260923 23:14:51.616786 763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:51.606883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:34607 (local address 127.24.227.194:36263)
0923 23:14:51.607023 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:51.607054 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:51.607474 (+ 420us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:51.607747 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:51.607750 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:51.608054 (+ 304us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:14:51.608284 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:51.608289 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:51.609346 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:51.609346 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:51.610274 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:51.610276 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:51.610662 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:51.610663 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:51.610790 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:51.611317 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:14:51.611327 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:14:51.613829 (+ 2502us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:14:51.615590 (+ 1761us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:51.615596 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:51.615597 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:51.615798 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:51.616015 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:51.616016 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:51.616016 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:51.616127 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:51.616327 (+ 200us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:14:51.616331 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:14:51.616430 (+ 99us) client_negotiation.cc:769] Sending connection context
0923 23:14:51.616507 (+ 77us) client_negotiation.cc:240] Negotiation successful
0923 23:14:51.616612 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":74,"threads_started":1}
I20260923 23:14:51.617276 870 heartbeater.cc:347] Connected to a master server at 127.24.227.254:34607
I20260923 23:14:51.617365 870 heartbeater.cc:464] Registering TS with master...
I20260923 23:14:51.617529 870 heartbeater.cc:511] Master 127.24.227.254:34607 requested a full tablet report, sending...
I20260923 23:14:51.617944 526 ts_manager.cc:194] Registered new tserver with Master: 9d239c22857640e7b2ff0c7008fbfaca (127.24.227.194:38277)
I20260923 23:14:51.618449 526 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.227.194@KRBTEST.COM'} at 127.24.227.194:36263
I20260923 23:14:51.619841 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 739
I20260923 23:14:51.619968 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/minicluster-data/ts-1/wal/instance
I20260923 23:14:51.620167 25487 external_mini_cluster.cc:1428] Setting key 9b4122f82d42912ce63b8407441e6fb0
I20260923 23:14:51.624104 596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:51.618246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.194:51849 (local address 127.24.227.254:34607)
0923 23:14:51.618284 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:51.618286 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:51.618350 (+ 64us) server_negotiation.cc:407] Connection header received
0923 23:14:51.618404 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:51.618404 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:51.618412 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:51.618450 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:51.618759 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:51.619390 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:51.620367 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:51.620571 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:51.620705 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:51.621528 (+ 823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:51.621529 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:51.621530 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:51.621536 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:51.622935 (+ 1399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:51.623312 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:51.623313 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:51.623314 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:51.623372 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:51.623568 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:51.623569 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:51.623569 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:51.623722 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:51.623799 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:51.623957 (+ 158us) server_negotiation.cc:299] Negotiation successful
0923 23:14:51.624021 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260923 23:14:51.624128 876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:51.618161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:34607 (local address 127.24.227.194:51849)
0923 23:14:51.618291 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:51.618300 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:51.618354 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:51.618463 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:51.618463 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:51.618645 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:14:51.618711 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:51.618712 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:51.619473 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:51.619474 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:51.620262 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:51.620264 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:51.620707 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:51.620708 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:51.620774 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:51.621193 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:14:51.621197 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:14:51.621440 (+ 243us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:14:51.623035 (+ 1595us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:51.623036 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:51.623037 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:51.623239 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:51.623438 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:51.623439 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:51.623440 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:51.623495 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:51.623802 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:14:51.623803 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:14:51.623845 (+ 42us) client_negotiation.cc:769] Sending connection context
0923 23:14:51.623924 (+ 79us) client_negotiation.cc:240] Negotiation successful
0923 23:14:51.624020 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":84,"threads_started":1}
WARNING: no policy specified for oryx/127.24.227.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.227.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.227.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/krb5kdc/oryx_127.24.227.195.keytab.
Entry for principal oryx/127.24.227.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/krb5kdc/oryx_127.24.227.195.keytab.
I20260923 23:14:51.644381 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.227.195:0
--local_ip_for_outbound_sockets=127.24.227.195
--webserver_interface=127.24.227.195
--webserver_port=0
--tserver_master_addrs=127.24.227.254:34607
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/krb5kdc/oryx_127.24.227.195.keytab
--principal=oryx/127.24.227.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:35707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:14:51.719135 879 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:14:51.719314 879 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:14:51.719336 879 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:14:51.720431 879 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:14:51.720469 879 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:14:51.720489 879 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260923 23:14:51.720539 879 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.227.195
I20260923 23:14:51.721909 879 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:35707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/krb5kdc/oryx_127.24.227.195.keytab
--principal=oryx/127.24.227.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.227.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.227.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.227.254:34607
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.879
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.227.195
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:14:51.722119 879 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:14:51.722337 879 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:14:51.726936 884 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:14:51.727036 879 server_base.cc:1034] running on GCE node
W20260923 23:14:51.726979 885 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:51.727161 887 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:14:51.727382 879 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:14:51.727602 879 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:14:51.728739 879 hybrid_clock.cc:648] HybridClock initialized: now 1790205291728718 us; error 33 us; skew 500 ppm
Sep 23 23:14:51 dist-test-slave-754n krb5kdc[482](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205291, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.227.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:14:51.731189 879 init.cc:377] Logged in from keytab as oryx/127.24.227.195@KRBTEST.COM (short username oryx)
I20260923 23:14:51.731657 879 webserver.cc:460] Webserver started at http://127.24.227.195:33259/ using document root <none> and password file <none>
I20260923 23:14:51.731956 879 fs_manager.cc:359] Metadata directory not provided
I20260923 23:14:51.732023 879 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:14:51.732172 879 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:14:51.733206 879 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/minicluster-data/ts-2/data/instance:
uuid: "e9d6ab43fc0f491daef44c6e09be0e4a"
format_stamp: "Formatted at 2026-09-23 23:14:51 on dist-test-slave-754n"
server_key: "8f89f367670b162143726c651cf5cff5"
server_key_iv: "e0e83a3428c39be6c1b4b2a2d8a9b445"
server_key_version: "encryptionkey@0"
I20260923 23:14:51.733569 879 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/minicluster-data/ts-2/wal/instance:
uuid: "e9d6ab43fc0f491daef44c6e09be0e4a"
format_stamp: "Formatted at 2026-09-23 23:14:51 on dist-test-slave-754n"
server_key: "8f89f367670b162143726c651cf5cff5"
server_key_iv: "e0e83a3428c39be6c1b4b2a2d8a9b445"
server_key_version: "encryptionkey@0"
I20260923 23:14:51.734777 879 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 23:14:51.735494 894 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:51.735663 879 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 23:14:51.735762 879 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/minicluster-data/ts-2/wal
uuid: "e9d6ab43fc0f491daef44c6e09be0e4a"
format_stamp: "Formatted at 2026-09-23 23:14:51 on dist-test-slave-754n"
server_key: "8f89f367670b162143726c651cf5cff5"
server_key_iv: "e0e83a3428c39be6c1b4b2a2d8a9b445"
server_key_version: "encryptionkey@0"
I20260923 23:14:51.735842 879 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:14:51.750227 879 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:14:51.752422 879 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:14:51.752578 879 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:14:51.753085 879 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 23:14:51.753141 879 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:51.753201 879 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 23:14:51.753232 879 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 23 23:14:51 dist-test-slave-754n krb5kdc[482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205291, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.227.195@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM
I20260923 23:14:51.760290 879 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.195:39441
I20260923 23:14:51.760398 1009 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.195:39441 every 8 connection(s)
I20260923 23:14:51.760670 879 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/minicluster-data/ts-2/data/info.pb
I20260923 23:14:51.762549 596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:51.753471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.195:52157 (local address 127.24.227.254:34607)
0923 23:14:51.753546 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:51.753549 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:51.753794 (+ 245us) server_negotiation.cc:407] Connection header received
0923 23:14:51.754208 (+ 414us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:51.754209 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:51.754222 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:51.754296 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:51.754998 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:51.755820 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:51.756874 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:51.757082 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:51.757173 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:51.759815 (+ 2642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:51.759817 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:51.759817 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:51.759825 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:51.761565 (+ 1740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:51.761898 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:51.761899 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:51.761899 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:51.761946 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:51.762098 (+ 152us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:51.762098 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:51.762098 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:51.762182 (+ 84us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:51.762230 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:51.762436 (+ 206us) server_negotiation.cc:299] Negotiation successful
0923 23:14:51.762488 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260923 23:14:51.762769 903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:51.753480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:34607 (local address 127.24.227.195:52157)
0923 23:14:51.753670 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:51.753708 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:51.754120 (+ 412us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:51.754349 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:51.754352 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:51.754670 (+ 318us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:14:51.754903 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:51.754908 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:51.755924 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:51.755925 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:51.756769 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:51.756771 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:51.757181 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:51.757182 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:51.757343 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:51.757806 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:14:51.757814 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:14:51.759717 (+ 1903us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:14:51.761631 (+ 1914us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:51.761636 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:51.761638 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:51.761837 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:51.761989 (+ 152us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:51.761989 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:51.761990 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:51.762066 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:51.762236 (+ 170us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:14:51.762240 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:14:51.762339 (+ 99us) client_negotiation.cc:769] Sending connection context
0923 23:14:51.762421 (+ 82us) client_negotiation.cc:240] Negotiation successful
0923 23:14:51.762525 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":92,"threads_started":1}
I20260923 23:14:51.763226 1010 heartbeater.cc:347] Connected to a master server at 127.24.227.254:34607
I20260923 23:14:51.763306 1010 heartbeater.cc:464] Registering TS with master...
I20260923 23:14:51.763473 1010 heartbeater.cc:511] Master 127.24.227.254:34607 requested a full tablet report, sending...
I20260923 23:14:51.763883 527 ts_manager.cc:194] Registered new tserver with Master: e9d6ab43fc0f491daef44c6e09be0e4a (127.24.227.195:39441)
I20260923 23:14:51.764245 527 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.227.195@KRBTEST.COM'} at 127.24.227.195:52157
I20260923 23:14:51.768291 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 879
I20260923 23:14:51.768411 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/minicluster-data/ts-2/wal/instance
I20260923 23:14:51.768620 25487 external_mini_cluster.cc:1428] Setting key a5a3d94d4d213c0b6958464f36dfe5df
I20260923 23:14:51.769439 25487 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 23:14:51.769538 596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:51.764293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.195:40079 (local address 127.24.227.254:34607)
0923 23:14:51.764368 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:51.764370 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:51.764505 (+ 135us) server_negotiation.cc:407] Connection header received
0923 23:14:51.764553 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:51.764553 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:51.764563 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:51.764620 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:51.764891 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:51.765393 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:51.766059 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:51.766235 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:51.766281 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:51.767098 (+ 817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:51.767099 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:51.767099 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:51.767104 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:51.768443 (+ 1339us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:51.768773 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:51.768774 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:51.768774 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:51.768829 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:51.769031 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:51.769032 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:51.769033 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:51.769154 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:51.769226 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:51.769380 (+ 154us) server_negotiation.cc:299] Negotiation successful
0923 23:14:51.769447 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260923 23:14:51.769546 1016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:51.764235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:34607 (local address 127.24.227.195:40079)
0923 23:14:51.764435 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:51.764445 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:51.764501 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:51.764631 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:51.764631 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:51.764777 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:14:51.764823 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:51.764824 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:51.765482 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:51.765482 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:51.765990 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:51.765991 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:51.766295 (+ 304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:51.766296 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:51.766341 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:51.766759 (+ 418us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:14:51.766764 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:14:51.767024 (+ 260us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:14:51.768510 (+ 1486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:51.768511 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:51.768512 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:51.768709 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:51.768899 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:51.768900 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:51.768900 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:51.768966 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:51.769232 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:14:51.769232 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:14:51.769274 (+ 42us) client_negotiation.cc:769] Sending connection context
0923 23:14:51.769356 (+ 82us) client_negotiation.cc:240] Negotiation successful
0923 23:14:51.769449 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":78,"threads_started":1}
Sep 23 23:14:51 dist-test-slave-754n krb5kdc[482](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205291, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 23 23:14:51 dist-test-slave-754n krb5kdc[482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 23:14:51 dist-test-slave-754n krb5kdc[482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
W20260923 23:14:51.789269 596 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 23:14:51.784591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58822 (local address 127.24.227.254:34607)
0923 23:14:51.784651 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:51.784653 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:51.784682 (+ 29us) server_negotiation.cc:407] Connection header received
0923 23:14:51.784699 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:51.784700 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:51.784707 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:51.784745 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:51.785261 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:51.785720 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:51.786399 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:51.786564 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:51.786612 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:51.789191 (+ 2579us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58822: BlockingRecv error: recv got EOF from 127.0.0.1:58822 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
Sep 23 23:14:51 dist-test-slave-754n krb5kdc[482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205291, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM
I20260923 23:14:51.795715 596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:51.789925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58832 (local address 127.24.227.254:34607)
0923 23:14:51.789959 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:51.789961 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:51.790014 (+ 53us) server_negotiation.cc:407] Connection header received
0923 23:14:51.790057 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:51.790058 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:51.790064 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:51.790120 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:51.790513 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:51.790927 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:51.791522 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:51.791677 (+ 155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:51.791725 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:51.793592 (+ 1867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:51.793593 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:51.793593 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:51.793599 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:51.794805 (+ 1206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:51.795135 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:51.795136 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:51.795136 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:51.795178 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:51.795360 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:51.795380 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:51.795380 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:51.795458 (+ 78us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:51.795519 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:51.795601 (+ 82us) server_negotiation.cc:299] Negotiation successful
0923 23:14:51.795646 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260923 23:14:51.797211 527 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58832:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 23:14:51.798662 527 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 23:14:51.804265 1032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:51.800797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.193:43785 (local address 127.0.0.1:57858)
0923 23:14:51.801181 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:51.801191 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:51.801262 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:51.801425 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:51.801425 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:51.801427 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:14:51.801458 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:51.801459 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:51.802345 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:51.802345 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:51.803087 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:51.803089 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:51.803985 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:51.803986 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:51.804061 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:51.804074 (+ 13us) client_negotiation.cc:769] Sending connection context
0923 23:14:51.804147 (+ 73us) client_negotiation.cc:240] Negotiation successful
0923 23:14:51.804193 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":358,"mutex_wait_us":44,"thread_start_us":54,"threads_started":1}
I20260923 23:14:51.804294 1033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:51.800909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57858 (local address 127.24.227.193:43785)
0923 23:14:51.801061 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:51.801065 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:51.801275 (+ 210us) server_negotiation.cc:407] Connection header received
0923 23:14:51.801319 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:51.801321 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:51.801336 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:51.801418 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:14:51.801514 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:51.802236 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:51.803190 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:51.803917 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:51.804001 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:51.804053 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:51.804163 (+ 110us) server_negotiation.cc:299] Negotiation successful
0923 23:14:51.804201 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":75,"threads_started":1}
I20260923 23:14:51.804773 1029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:51.800594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.195:39441 (local address 127.0.0.1:43294)
0923 23:14:51.800734 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:51.800769 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:51.800831 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:51.801342 (+ 511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:51.801345 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:51.801346 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:14:51.801433 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:51.801435 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:51.802282 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:51.802283 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:51.803470 (+ 1187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:51.803473 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:51.804545 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:51.804546 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:51.804618 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:51.804622 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 23:14:51.804674 (+ 52us) client_negotiation.cc:240] Negotiation successful
0923 23:14:51.804710 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"mutex_wait_us":22,"thread_start_us":48,"threads_started":1}
I20260923 23:14:51.804999 1034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:51.800911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43294 (local address 127.24.227.195:39441)
0923 23:14:51.801119 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:51.801122 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:51.801148 (+ 26us) server_negotiation.cc:407] Connection header received
0923 23:14:51.801184 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:51.801186 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:51.801206 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:51.801276 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:14:51.801508 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:51.802206 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:51.803554 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:51.804378 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:51.804467 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:51.804515 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:51.804883 (+ 368us) server_negotiation.cc:299] Negotiation successful
0923 23:14:51.804918 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":76,"threads_started":1}
I20260923 23:14:51.805308 664 tablet_service.cc:1467] Processing CreateTablet for tablet 91ea62596e2a4ad4984dd3642df1912b (DEFAULT_TABLE table=test-table [id=e4769796168b4ac29069ed2157b09727]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 23:14:51.805635 664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 91ea62596e2a4ad4984dd3642df1912b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:14:51.805546 944 tablet_service.cc:1467] Processing CreateTablet for tablet 91ea62596e2a4ad4984dd3642df1912b (DEFAULT_TABLE table=test-table [id=e4769796168b4ac29069ed2157b09727]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 23:14:51.805882 944 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 91ea62596e2a4ad4984dd3642df1912b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:14:51.806119 1031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:51.800745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35736 (local address 127.24.227.194:38277)
0923 23:14:51.801159 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:51.801165 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:51.801174 (+ 9us) server_negotiation.cc:407] Connection header received
0923 23:14:51.801217 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:51.801219 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:51.801233 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:51.801326 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:14:51.802355 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:51.803397 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:51.804659 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:51.805540 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:51.805631 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:51.805678 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:51.805988 (+ 310us) server_negotiation.cc:299] Negotiation successful
0923 23:14:51.806027 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":75,"threads_started":1}
I20260923 23:14:51.806253 1030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:51.800624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.194:38277 (local address 127.0.0.1:35736)
0923 23:14:51.801057 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:51.801067 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:51.801120 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:51.801394 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:51.801395 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:51.801397 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:14:51.801456 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:51.801458 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:51.803526 (+ 2068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:51.803527 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:51.804565 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:51.804568 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:51.805626 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:51.805627 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:51.805771 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:51.805775 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 23:14:51.805828 (+ 53us) client_negotiation.cc:240] Negotiation successful
0923 23:14:51.805858 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":55,"threads_started":1}
I20260923 23:14:51.807081 804 tablet_service.cc:1467] Processing CreateTablet for tablet 91ea62596e2a4ad4984dd3642df1912b (DEFAULT_TABLE table=test-table [id=e4769796168b4ac29069ed2157b09727]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 23:14:51.807417 804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 91ea62596e2a4ad4984dd3642df1912b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:14:51.809376 1035 tablet_bootstrap.cc:492] T 91ea62596e2a4ad4984dd3642df1912b P e9d6ab43fc0f491daef44c6e09be0e4a: Bootstrap starting.
I20260923 23:14:51.809655 1036 tablet_bootstrap.cc:492] T 91ea62596e2a4ad4984dd3642df1912b P 2cd0dd070b8643bf966ede03f18a5de4: Bootstrap starting.
I20260923 23:14:51.810025 1035 tablet_bootstrap.cc:654] T 91ea62596e2a4ad4984dd3642df1912b P e9d6ab43fc0f491daef44c6e09be0e4a: Neither blocks nor log segments found. Creating new log.
I20260923 23:14:51.810189 1036 tablet_bootstrap.cc:654] T 91ea62596e2a4ad4984dd3642df1912b P 2cd0dd070b8643bf966ede03f18a5de4: Neither blocks nor log segments found. Creating new log.
I20260923 23:14:51.810307 1035 log.cc:824] T 91ea62596e2a4ad4984dd3642df1912b P e9d6ab43fc0f491daef44c6e09be0e4a: Log is configured to *not* fsync() on all Append() calls
I20260923 23:14:51.810472 1036 log.cc:824] T 91ea62596e2a4ad4984dd3642df1912b P 2cd0dd070b8643bf966ede03f18a5de4: Log is configured to *not* fsync() on all Append() calls
I20260923 23:14:51.810963 1039 tablet_bootstrap.cc:492] T 91ea62596e2a4ad4984dd3642df1912b P 9d239c22857640e7b2ff0c7008fbfaca: Bootstrap starting.
I20260923 23:14:51.811015 1035 tablet_bootstrap.cc:492] T 91ea62596e2a4ad4984dd3642df1912b P e9d6ab43fc0f491daef44c6e09be0e4a: No bootstrap required, opened a new log
I20260923 23:14:51.811074 1035 ts_tablet_manager.cc:1397] T 91ea62596e2a4ad4984dd3642df1912b P e9d6ab43fc0f491daef44c6e09be0e4a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 23:14:51.811293 1036 tablet_bootstrap.cc:492] T 91ea62596e2a4ad4984dd3642df1912b P 2cd0dd070b8643bf966ede03f18a5de4: No bootstrap required, opened a new log
I20260923 23:14:51.811370 1036 ts_tablet_manager.cc:1397] T 91ea62596e2a4ad4984dd3642df1912b P 2cd0dd070b8643bf966ede03f18a5de4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 23:14:51.811547 1039 tablet_bootstrap.cc:654] T 91ea62596e2a4ad4984dd3642df1912b P 9d239c22857640e7b2ff0c7008fbfaca: Neither blocks nor log segments found. Creating new log.
I20260923 23:14:51.811847 1039 log.cc:824] T 91ea62596e2a4ad4984dd3642df1912b P 9d239c22857640e7b2ff0c7008fbfaca: Log is configured to *not* fsync() on all Append() calls
I20260923 23:14:51.812556 1039 tablet_bootstrap.cc:492] T 91ea62596e2a4ad4984dd3642df1912b P 9d239c22857640e7b2ff0c7008fbfaca: No bootstrap required, opened a new log
I20260923 23:14:51.812631 1039 ts_tablet_manager.cc:1397] T 91ea62596e2a4ad4984dd3642df1912b P 9d239c22857640e7b2ff0c7008fbfaca: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 23:14:51.813056 1035 raft_consensus.cc:348] T 91ea62596e2a4ad4984dd3642df1912b P e9d6ab43fc0f491daef44c6e09be0e4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9d6ab43fc0f491daef44c6e09be0e4a" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39441 } } peers { permanent_uuid: "9d239c22857640e7b2ff0c7008fbfaca" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 38277 } } peers { permanent_uuid: "2cd0dd070b8643bf966ede03f18a5de4" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 43785 } }
I20260923 23:14:51.813251 1035 raft_consensus.cc:374] T 91ea62596e2a4ad4984dd3642df1912b P e9d6ab43fc0f491daef44c6e09be0e4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:14:51.813287 1035 raft_consensus.cc:729] T 91ea62596e2a4ad4984dd3642df1912b P e9d6ab43fc0f491daef44c6e09be0e4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9d6ab43fc0f491daef44c6e09be0e4a, State: Initialized, Role: FOLLOWER
I20260923 23:14:51.813413 1035 consensus_queue.cc:260] T 91ea62596e2a4ad4984dd3642df1912b P e9d6ab43fc0f491daef44c6e09be0e4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9d6ab43fc0f491daef44c6e09be0e4a" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39441 } } peers { permanent_uuid: "9d239c22857640e7b2ff0c7008fbfaca" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 38277 } } peers { permanent_uuid: "2cd0dd070b8643bf966ede03f18a5de4" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 43785 } }
I20260923 23:14:51.813395 1036 raft_consensus.cc:348] T 91ea62596e2a4ad4984dd3642df1912b P 2cd0dd070b8643bf966ede03f18a5de4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9d6ab43fc0f491daef44c6e09be0e4a" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39441 } } peers { permanent_uuid: "9d239c22857640e7b2ff0c7008fbfaca" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 38277 } } peers { permanent_uuid: "2cd0dd070b8643bf966ede03f18a5de4" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 43785 } }
I20260923 23:14:51.813544 1036 raft_consensus.cc:374] T 91ea62596e2a4ad4984dd3642df1912b P 2cd0dd070b8643bf966ede03f18a5de4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:14:51.813578 1036 raft_consensus.cc:729] T 91ea62596e2a4ad4984dd3642df1912b P 2cd0dd070b8643bf966ede03f18a5de4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cd0dd070b8643bf966ede03f18a5de4, State: Initialized, Role: FOLLOWER
I20260923 23:14:51.813660 1035 ts_tablet_manager.cc:1428] T 91ea62596e2a4ad4984dd3642df1912b P e9d6ab43fc0f491daef44c6e09be0e4a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 23:14:51.813724 1036 consensus_queue.cc:260] T 91ea62596e2a4ad4984dd3642df1912b P 2cd0dd070b8643bf966ede03f18a5de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9d6ab43fc0f491daef44c6e09be0e4a" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39441 } } peers { permanent_uuid: "9d239c22857640e7b2ff0c7008fbfaca" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 38277 } } peers { permanent_uuid: "2cd0dd070b8643bf966ede03f18a5de4" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 43785 } }
I20260923 23:14:51.813818 1010 heartbeater.cc:503] Master 127.24.227.254:34607 was elected leader, sending a full tablet report...
I20260923 23:14:51.813956 1036 ts_tablet_manager.cc:1428] T 91ea62596e2a4ad4984dd3642df1912b P 2cd0dd070b8643bf966ede03f18a5de4: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 23:14:51.814111 730 heartbeater.cc:503] Master 127.24.227.254:34607 was elected leader, sending a full tablet report...
I20260923 23:14:51.814539 1039 raft_consensus.cc:348] T 91ea62596e2a4ad4984dd3642df1912b P 9d239c22857640e7b2ff0c7008fbfaca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9d6ab43fc0f491daef44c6e09be0e4a" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39441 } } peers { permanent_uuid: "9d239c22857640e7b2ff0c7008fbfaca" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 38277 } } peers { permanent_uuid: "2cd0dd070b8643bf966ede03f18a5de4" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 43785 } }
I20260923 23:14:51.814693 1039 raft_consensus.cc:374] T 91ea62596e2a4ad4984dd3642df1912b P 9d239c22857640e7b2ff0c7008fbfaca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:14:51.814736 1039 raft_consensus.cc:729] T 91ea62596e2a4ad4984dd3642df1912b P 9d239c22857640e7b2ff0c7008fbfaca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d239c22857640e7b2ff0c7008fbfaca, State: Initialized, Role: FOLLOWER
I20260923 23:14:51.814837 1039 consensus_queue.cc:260] T 91ea62596e2a4ad4984dd3642df1912b P 9d239c22857640e7b2ff0c7008fbfaca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9d6ab43fc0f491daef44c6e09be0e4a" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39441 } } peers { permanent_uuid: "9d239c22857640e7b2ff0c7008fbfaca" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 38277 } } peers { permanent_uuid: "2cd0dd070b8643bf966ede03f18a5de4" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 43785 } }
I20260923 23:14:51.815066 1039 ts_tablet_manager.cc:1428] T 91ea62596e2a4ad4984dd3642df1912b P 9d239c22857640e7b2ff0c7008fbfaca: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260923 23:14:51.815125 870 heartbeater.cc:503] Master 127.24.227.254:34607 was elected leader, sending a full tablet report...
I20260923 23:14:51.823392 1043 raft_consensus.cc:482] T 91ea62596e2a4ad4984dd3642df1912b P 9d239c22857640e7b2ff0c7008fbfaca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 23:14:51.823458 1043 raft_consensus.cc:504] T 91ea62596e2a4ad4984dd3642df1912b P 9d239c22857640e7b2ff0c7008fbfaca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9d6ab43fc0f491daef44c6e09be0e4a" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39441 } } peers { permanent_uuid: "9d239c22857640e7b2ff0c7008fbfaca" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 38277 } } peers { permanent_uuid: "2cd0dd070b8643bf966ede03f18a5de4" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 43785 } }
I20260923 23:14:51.823769 1043 leader_election.cc:288] T 91ea62596e2a4ad4984dd3642df1912b P 9d239c22857640e7b2ff0c7008fbfaca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9d6ab43fc0f491daef44c6e09be0e4a (127.24.227.195:39441), 2cd0dd070b8643bf966ede03f18a5de4 (127.24.227.193:43785)
I20260923 23:14:51.827436 1034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:51.823990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.194:42307 (local address 127.24.227.195:39441)
0923 23:14:51.824026 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:51.824027 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:51.824162 (+ 135us) server_negotiation.cc:407] Connection header received
0923 23:14:51.824223 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:51.824224 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:51.824235 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:51.824275 (+ 40us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:14:51.824427 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:51.825086 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:51.826492 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:51.827025 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:51.827095 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:51.827100 (+ 5us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:51.827346 (+ 246us) server_negotiation.cc:299] Negotiation successful
0923 23:14:51.827377 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260923 23:14:51.827453 763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:51.823948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.193:43785 (local address 127.24.227.194:49977)
0923 23:14:51.823995 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:51.824006 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:51.824047 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:51.824190 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:51.824191 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:51.824192 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:14:51.824276 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:51.824277 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:51.825155 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:51.825156 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:51.826405 (+ 1249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:51.826407 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:51.827117 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:51.827117 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:51.827251 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:51.827256 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 23:14:51.827309 (+ 53us) client_negotiation.cc:240] Negotiation successful
0923 23:14:51.827362 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30,"mutex_wait_us":64}
I20260923 23:14:51.827485 1033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:51.823999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.194:49977 (local address 127.24.227.193:43785)
0923 23:14:51.824033 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:51.824035 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:51.824043 (+ 8us) server_negotiation.cc:407] Connection header received
0923 23:14:51.824102 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:51.824103 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:51.824109 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:51.824176 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:14:51.824339 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:51.825072 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:51.826493 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:51.827024 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:51.827093 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:51.827100 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:51.827317 (+ 217us) server_negotiation.cc:299] Negotiation successful
0923 23:14:51.827392 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260923 23:14:51.827468 1044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:51.823951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.195:39441 (local address 127.24.227.194:42307)
0923 23:14:51.824093 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:51.824102 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:51.824163 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:51.824295 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:51.824296 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:51.824297 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:14:51.824362 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:51.824363 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:51.825156 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:51.825156 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:51.826405 (+ 1249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:51.826407 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:51.827149 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:51.827150 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:51.827236 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:51.827239 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 23:14:51.827338 (+ 99us) client_negotiation.cc:240] Negotiation successful
0923 23:14:51.827377 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"mutex_wait_us":66,"thread_start_us":70,"threads_started":1}
I20260923 23:14:51.827915 684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91ea62596e2a4ad4984dd3642df1912b" candidate_uuid: "9d239c22857640e7b2ff0c7008fbfaca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cd0dd070b8643bf966ede03f18a5de4" is_pre_election: true
I20260923 23:14:51.828032 684 raft_consensus.cc:2454] T 91ea62596e2a4ad4984dd3642df1912b P 2cd0dd070b8643bf966ede03f18a5de4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d239c22857640e7b2ff0c7008fbfaca in term 0.
I20260923 23:14:51.828163 963 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91ea62596e2a4ad4984dd3642df1912b" candidate_uuid: "9d239c22857640e7b2ff0c7008fbfaca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9d6ab43fc0f491daef44c6e09be0e4a" is_pre_election: true
I20260923 23:14:51.828296 963 raft_consensus.cc:2454] T 91ea62596e2a4ad4984dd3642df1912b P e9d6ab43fc0f491daef44c6e09be0e4a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d239c22857640e7b2ff0c7008fbfaca in term 0.
I20260923 23:14:51.828517 755 leader_election.cc:302] T 91ea62596e2a4ad4984dd3642df1912b P 9d239c22857640e7b2ff0c7008fbfaca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d239c22857640e7b2ff0c7008fbfaca, e9d6ab43fc0f491daef44c6e09be0e4a; no voters:
I20260923 23:14:51.828658 1043 raft_consensus.cc:2781] T 91ea62596e2a4ad4984dd3642df1912b P 9d239c22857640e7b2ff0c7008fbfaca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 23:14:51.828702 1043 raft_consensus.cc:482] T 91ea62596e2a4ad4984dd3642df1912b P 9d239c22857640e7b2ff0c7008fbfaca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 23:14:51.828716 1043 raft_consensus.cc:3037] T 91ea62596e2a4ad4984dd3642df1912b P 9d239c22857640e7b2ff0c7008fbfaca [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:14:51.829473 1043 raft_consensus.cc:504] T 91ea62596e2a4ad4984dd3642df1912b P 9d239c22857640e7b2ff0c7008fbfaca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9d6ab43fc0f491daef44c6e09be0e4a" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39441 } } peers { permanent_uuid: "9d239c22857640e7b2ff0c7008fbfaca" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 38277 } } peers { permanent_uuid: "2cd0dd070b8643bf966ede03f18a5de4" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 43785 } }
I20260923 23:14:51.829624 1043 leader_election.cc:288] T 91ea62596e2a4ad4984dd3642df1912b P 9d239c22857640e7b2ff0c7008fbfaca [CANDIDATE]: Term 1 election: Requested vote from peers e9d6ab43fc0f491daef44c6e09be0e4a (127.24.227.195:39441), 2cd0dd070b8643bf966ede03f18a5de4 (127.24.227.193:43785)
I20260923 23:14:51.829766 963 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91ea62596e2a4ad4984dd3642df1912b" candidate_uuid: "9d239c22857640e7b2ff0c7008fbfaca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9d6ab43fc0f491daef44c6e09be0e4a"
I20260923 23:14:51.829783 684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91ea62596e2a4ad4984dd3642df1912b" candidate_uuid: "9d239c22857640e7b2ff0c7008fbfaca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cd0dd070b8643bf966ede03f18a5de4"
I20260923 23:14:51.829849 684 raft_consensus.cc:3037] T 91ea62596e2a4ad4984dd3642df1912b P 2cd0dd070b8643bf966ede03f18a5de4 [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:14:51.829833 963 raft_consensus.cc:3037] T 91ea62596e2a4ad4984dd3642df1912b P e9d6ab43fc0f491daef44c6e09be0e4a [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:14:51.830624 684 raft_consensus.cc:2454] T 91ea62596e2a4ad4984dd3642df1912b P 2cd0dd070b8643bf966ede03f18a5de4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d239c22857640e7b2ff0c7008fbfaca in term 1.
I20260923 23:14:51.830624 963 raft_consensus.cc:2454] T 91ea62596e2a4ad4984dd3642df1912b P e9d6ab43fc0f491daef44c6e09be0e4a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d239c22857640e7b2ff0c7008fbfaca in term 1.
I20260923 23:14:51.830803 757 leader_election.cc:302] T 91ea62596e2a4ad4984dd3642df1912b P 9d239c22857640e7b2ff0c7008fbfaca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cd0dd070b8643bf966ede03f18a5de4, 9d239c22857640e7b2ff0c7008fbfaca; no voters:
I20260923 23:14:51.830886 1043 raft_consensus.cc:2781] T 91ea62596e2a4ad4984dd3642df1912b P 9d239c22857640e7b2ff0c7008fbfaca [term 1 FOLLOWER]: Leader election won for term 1
I20260923 23:14:51.830993 1043 raft_consensus.cc:686] T 91ea62596e2a4ad4984dd3642df1912b P 9d239c22857640e7b2ff0c7008fbfaca [term 1 LEADER]: Becoming Leader. State: Replica: 9d239c22857640e7b2ff0c7008fbfaca, State: Running, Role: LEADER
I20260923 23:14:51.831101 1043 consensus_queue.cc:237] T 91ea62596e2a4ad4984dd3642df1912b P 9d239c22857640e7b2ff0c7008fbfaca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9d6ab43fc0f491daef44c6e09be0e4a" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39441 } } peers { permanent_uuid: "9d239c22857640e7b2ff0c7008fbfaca" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 38277 } } peers { permanent_uuid: "2cd0dd070b8643bf966ede03f18a5de4" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 43785 } }
I20260923 23:14:51.831744 527 catalog_manager.cc:5495] T 91ea62596e2a4ad4984dd3642df1912b P 9d239c22857640e7b2ff0c7008fbfaca reported cstate change: term changed from 0 to 1, leader changed from <none> to 9d239c22857640e7b2ff0c7008fbfaca (127.24.227.194). New cstate: current_term: 1 leader_uuid: "9d239c22857640e7b2ff0c7008fbfaca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9d6ab43fc0f491daef44c6e09be0e4a" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39441 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d239c22857640e7b2ff0c7008fbfaca" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 38277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2cd0dd070b8643bf966ede03f18a5de4" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 43785 } health_report { overall_health: UNKNOWN } } }
I20260923 23:14:51.839635 1031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:51.837483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35738 (local address 127.24.227.194:38277)
0923 23:14:51.837518 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:51.837520 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:51.837528 (+ 8us) server_negotiation.cc:407] Connection header received
0923 23:14:51.837571 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:51.837572 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:51.837586 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:51.837647 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 23:14:51.837784 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:51.838222 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:51.838911 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:51.839103 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:51.839183 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:51.839352 (+ 169us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 23:14:51.839436 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 23:14:51.839511 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:51.839549 (+ 38us) server_negotiation.cc:299] Negotiation successful
0923 23:14:51.839566 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260923 23:14:51.841048 963 raft_consensus.cc:1261] T 91ea62596e2a4ad4984dd3642df1912b P e9d6ab43fc0f491daef44c6e09be0e4a [term 1 FOLLOWER]: Refusing update from remote peer 9d239c22857640e7b2ff0c7008fbfaca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 23:14:51.841111 684 raft_consensus.cc:1261] T 91ea62596e2a4ad4984dd3642df1912b P 2cd0dd070b8643bf966ede03f18a5de4 [term 1 FOLLOWER]: Refusing update from remote peer 9d239c22857640e7b2ff0c7008fbfaca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 23:14:51.841323 1045 consensus_queue.cc:1035] T 91ea62596e2a4ad4984dd3642df1912b P 9d239c22857640e7b2ff0c7008fbfaca [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9d6ab43fc0f491daef44c6e09be0e4a" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39441 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 23:14:51.841418 1043 consensus_queue.cc:1035] T 91ea62596e2a4ad4984dd3642df1912b P 9d239c22857640e7b2ff0c7008fbfaca [LEADER]: Connected to new peer: Peer: permanent_uuid: "2cd0dd070b8643bf966ede03f18a5de4" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 43785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 23:14:51.843521 1049 mvcc.cc:205] Tried to move back new op lower bound from 7332680875378618368 to 7332680875340451840. Current Snapshot: MvccSnapshot[applied={T|T < 7332680875378618368}]
I20260923 23:14:51.844003 1050 mvcc.cc:205] Tried to move back new op lower bound from 7332680875378618368 to 7332680875340451840. Current Snapshot: MvccSnapshot[applied={T|T < 7332680875378618368}]
I20260923 23:14:51.845742 527 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58832:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 23:14:51.845808 527 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58832:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 23:14:51.846437 527 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 607f68c4a2fb4f858b83bbc07edd2bdb: Sending DeleteTablet for 3 replicas of tablet 91ea62596e2a4ad4984dd3642df1912b
I20260923 23:14:51.846712 944 tablet_service.cc:1514] Processing DeleteTablet for tablet 91ea62596e2a4ad4984dd3642df1912b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 23:14:51 UTC) from {username='oryx', principal='oryx/127.24.227.254@KRBTEST.COM'} at 127.0.0.1:43294
I20260923 23:14:51.846742 664 tablet_service.cc:1514] Processing DeleteTablet for tablet 91ea62596e2a4ad4984dd3642df1912b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 23:14:51 UTC) from {username='oryx', principal='oryx/127.24.227.254@KRBTEST.COM'} at 127.0.0.1:57858
I20260923 23:14:51.846880 527 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58832:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260923 23:14:51.846959 804 tablet_service.cc:1514] Processing DeleteTablet for tablet 91ea62596e2a4ad4984dd3642df1912b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 23:14:51 UTC) from {username='oryx', principal='oryx/127.24.227.254@KRBTEST.COM'} at 127.0.0.1:35736
W20260923 23:14:51.846998 527 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 23:14:51.847100 1058 tablet_replica.cc:330] T 91ea62596e2a4ad4984dd3642df1912b P 2cd0dd070b8643bf966ede03f18a5de4: stopping tablet replica
I20260923 23:14:51.847180 1058 raft_consensus.cc:2229] T 91ea62596e2a4ad4984dd3642df1912b P 2cd0dd070b8643bf966ede03f18a5de4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 23:14:51.847289 1058 raft_consensus.cc:2258] T 91ea62596e2a4ad4984dd3642df1912b P 2cd0dd070b8643bf966ede03f18a5de4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 23:14:51.847453 1057 tablet_replica.cc:330] T 91ea62596e2a4ad4984dd3642df1912b P e9d6ab43fc0f491daef44c6e09be0e4a: stopping tablet replica
I20260923 23:14:51.847536 1057 raft_consensus.cc:2229] T 91ea62596e2a4ad4984dd3642df1912b P e9d6ab43fc0f491daef44c6e09be0e4a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 23:14:51.847857 1057 raft_consensus.cc:2258] T 91ea62596e2a4ad4984dd3642df1912b P e9d6ab43fc0f491daef44c6e09be0e4a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 23:14:51.848284 1058 ts_tablet_manager.cc:1905] T 91ea62596e2a4ad4984dd3642df1912b P 2cd0dd070b8643bf966ede03f18a5de4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 23:14:51.848397 944 tablet_service.cc:1467] Processing CreateTablet for tablet eb2d2d755d374109a0c172e42d4d691f (DEFAULT_TABLE table=test-table [id=65cbc566eb29437daec5273774504e55]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 23:14:51.848462 664 tablet_service.cc:1467] Processing CreateTablet for tablet eb2d2d755d374109a0c172e42d4d691f (DEFAULT_TABLE table=test-table [id=65cbc566eb29437daec5273774504e55]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 23:14:51.848493 944 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb2d2d755d374109a0c172e42d4d691f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:14:51.848543 664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb2d2d755d374109a0c172e42d4d691f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:14:51.849634 1036 tablet_bootstrap.cc:492] T eb2d2d755d374109a0c172e42d4d691f P 2cd0dd070b8643bf966ede03f18a5de4: Bootstrap starting.
I20260923 23:14:51.849661 1035 tablet_bootstrap.cc:492] T eb2d2d755d374109a0c172e42d4d691f P e9d6ab43fc0f491daef44c6e09be0e4a: Bootstrap starting.
I20260923 23:14:51.849858 1058 ts_tablet_manager.cc:1918] T 91ea62596e2a4ad4984dd3642df1912b P 2cd0dd070b8643bf966ede03f18a5de4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 23:14:51.849933 1058 log.cc:1196] T 91ea62596e2a4ad4984dd3642df1912b P 2cd0dd070b8643bf966ede03f18a5de4: Deleting WAL directory at /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/minicluster-data/ts-0/wal/wals/91ea62596e2a4ad4984dd3642df1912b
I20260923 23:14:51.850026 1036 tablet_bootstrap.cc:654] T eb2d2d755d374109a0c172e42d4d691f P 2cd0dd070b8643bf966ede03f18a5de4: Neither blocks nor log segments found. Creating new log.
I20260923 23:14:51.850088 1035 tablet_bootstrap.cc:654] T eb2d2d755d374109a0c172e42d4d691f P e9d6ab43fc0f491daef44c6e09be0e4a: Neither blocks nor log segments found. Creating new log.
I20260923 23:14:51.850220 1058 ts_tablet_manager.cc:1939] T 91ea62596e2a4ad4984dd3642df1912b P 2cd0dd070b8643bf966ede03f18a5de4: Deleting consensus metadata
I20260923 23:14:51.850478 804 tablet_service.cc:1467] Processing CreateTablet for tablet eb2d2d755d374109a0c172e42d4d691f (DEFAULT_TABLE table=test-table [id=65cbc566eb29437daec5273774504e55]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 23:14:51.850564 513 catalog_manager.cc:4841] TS 2cd0dd070b8643bf966ede03f18a5de4 (127.24.227.193:43785): tablet 91ea62596e2a4ad4984dd3642df1912b (table test-table [id=e4769796168b4ac29069ed2157b09727]) successfully deleted
I20260923 23:14:51.850590 804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb2d2d755d374109a0c172e42d4d691f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:14:51.850644 1035 tablet_bootstrap.cc:492] T eb2d2d755d374109a0c172e42d4d691f P e9d6ab43fc0f491daef44c6e09be0e4a: No bootstrap required, opened a new log
I20260923 23:14:51.850692 1035 ts_tablet_manager.cc:1397] T eb2d2d755d374109a0c172e42d4d691f P e9d6ab43fc0f491daef44c6e09be0e4a: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260923 23:14:51.850852 1035 raft_consensus.cc:348] T eb2d2d755d374109a0c172e42d4d691f P e9d6ab43fc0f491daef44c6e09be0e4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cd0dd070b8643bf966ede03f18a5de4" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 43785 } } peers { permanent_uuid: "9d239c22857640e7b2ff0c7008fbfaca" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 38277 } } peers { permanent_uuid: "e9d6ab43fc0f491daef44c6e09be0e4a" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39441 } }
I20260923 23:14:51.850935 1035 raft_consensus.cc:374] T eb2d2d755d374109a0c172e42d4d691f P e9d6ab43fc0f491daef44c6e09be0e4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:14:51.850961 1035 raft_consensus.cc:729] T eb2d2d755d374109a0c172e42d4d691f P e9d6ab43fc0f491daef44c6e09be0e4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9d6ab43fc0f491daef44c6e09be0e4a, State: Initialized, Role: FOLLOWER
I20260923 23:14:51.851001 1035 consensus_queue.cc:260] T eb2d2d755d374109a0c172e42d4d691f P e9d6ab43fc0f491daef44c6e09be0e4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cd0dd070b8643bf966ede03f18a5de4" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 43785 } } peers { permanent_uuid: "9d239c22857640e7b2ff0c7008fbfaca" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 38277 } } peers { permanent_uuid: "e9d6ab43fc0f491daef44c6e09be0e4a" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39441 } }
I20260923 23:14:51.851086 1035 ts_tablet_manager.cc:1428] T eb2d2d755d374109a0c172e42d4d691f P e9d6ab43fc0f491daef44c6e09be0e4a: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260923 23:14:51.851199 1036 tablet_bootstrap.cc:492] T eb2d2d755d374109a0c172e42d4d691f P 2cd0dd070b8643bf966ede03f18a5de4: No bootstrap required, opened a new log
I20260923 23:14:51.850881 1057 ts_tablet_manager.cc:1905] T 91ea62596e2a4ad4984dd3642df1912b P e9d6ab43fc0f491daef44c6e09be0e4a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 23:14:51.851243 1036 ts_tablet_manager.cc:1397] T eb2d2d755d374109a0c172e42d4d691f P 2cd0dd070b8643bf966ede03f18a5de4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 23:14:51.851383 1036 raft_consensus.cc:348] T eb2d2d755d374109a0c172e42d4d691f P 2cd0dd070b8643bf966ede03f18a5de4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cd0dd070b8643bf966ede03f18a5de4" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 43785 } } peers { permanent_uuid: "9d239c22857640e7b2ff0c7008fbfaca" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 38277 } } peers { permanent_uuid: "e9d6ab43fc0f491daef44c6e09be0e4a" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39441 } }
I20260923 23:14:51.851446 1036 raft_consensus.cc:374] T eb2d2d755d374109a0c172e42d4d691f P 2cd0dd070b8643bf966ede03f18a5de4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:14:51.851466 1036 raft_consensus.cc:729] T eb2d2d755d374109a0c172e42d4d691f P 2cd0dd070b8643bf966ede03f18a5de4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cd0dd070b8643bf966ede03f18a5de4, State: Initialized, Role: FOLLOWER
I20260923 23:14:51.851503 1036 consensus_queue.cc:260] T eb2d2d755d374109a0c172e42d4d691f P 2cd0dd070b8643bf966ede03f18a5de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cd0dd070b8643bf966ede03f18a5de4" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 43785 } } peers { permanent_uuid: "9d239c22857640e7b2ff0c7008fbfaca" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 38277 } } peers { permanent_uuid: "e9d6ab43fc0f491daef44c6e09be0e4a" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39441 } }
I20260923 23:14:51.851614 1036 ts_tablet_manager.cc:1428] T eb2d2d755d374109a0c172e42d4d691f P 2cd0dd070b8643bf966ede03f18a5de4: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:51.851826 1059 tablet_replica.cc:330] T 91ea62596e2a4ad4984dd3642df1912b P 9d239c22857640e7b2ff0c7008fbfaca: stopping tablet replica
I20260923 23:14:51.851926 1059 raft_consensus.cc:2229] T 91ea62596e2a4ad4984dd3642df1912b P 9d239c22857640e7b2ff0c7008fbfaca [term 1 LEADER]: Raft consensus shutting down.
I20260923 23:14:51.851964 1039 tablet_bootstrap.cc:492] T eb2d2d755d374109a0c172e42d4d691f P 9d239c22857640e7b2ff0c7008fbfaca: Bootstrap starting.
I20260923 23:14:51.852025 1059 raft_consensus.cc:2258] T 91ea62596e2a4ad4984dd3642df1912b P 9d239c22857640e7b2ff0c7008fbfaca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 23:14:51.852352 1039 tablet_bootstrap.cc:654] T eb2d2d755d374109a0c172e42d4d691f P 9d239c22857640e7b2ff0c7008fbfaca: Neither blocks nor log segments found. Creating new log.
I20260923 23:14:51.852464 1059 ts_tablet_manager.cc:1905] T 91ea62596e2a4ad4984dd3642df1912b P 9d239c22857640e7b2ff0c7008fbfaca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 23:14:51.852697 1057 ts_tablet_manager.cc:1918] T 91ea62596e2a4ad4984dd3642df1912b P e9d6ab43fc0f491daef44c6e09be0e4a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 23:14:51.852769 1057 log.cc:1196] T 91ea62596e2a4ad4984dd3642df1912b P e9d6ab43fc0f491daef44c6e09be0e4a: Deleting WAL directory at /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/minicluster-data/ts-2/wal/wals/91ea62596e2a4ad4984dd3642df1912b
I20260923 23:14:51.852999 1057 ts_tablet_manager.cc:1939] T 91ea62596e2a4ad4984dd3642df1912b P e9d6ab43fc0f491daef44c6e09be0e4a: Deleting consensus metadata
I20260923 23:14:51.853001 1039 tablet_bootstrap.cc:492] T eb2d2d755d374109a0c172e42d4d691f P 9d239c22857640e7b2ff0c7008fbfaca: No bootstrap required, opened a new log
I20260923 23:14:51.853040 1039 ts_tablet_manager.cc:1397] T eb2d2d755d374109a0c172e42d4d691f P 9d239c22857640e7b2ff0c7008fbfaca: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260923 23:14:51.853231 1039 raft_consensus.cc:348] T eb2d2d755d374109a0c172e42d4d691f P 9d239c22857640e7b2ff0c7008fbfaca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cd0dd070b8643bf966ede03f18a5de4" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 43785 } } peers { permanent_uuid: "9d239c22857640e7b2ff0c7008fbfaca" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 38277 } } peers { permanent_uuid: "e9d6ab43fc0f491daef44c6e09be0e4a" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39441 } }
I20260923 23:14:51.853320 1039 raft_consensus.cc:374] T eb2d2d755d374109a0c172e42d4d691f P 9d239c22857640e7b2ff0c7008fbfaca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:14:51.853346 1039 raft_consensus.cc:729] T eb2d2d755d374109a0c172e42d4d691f P 9d239c22857640e7b2ff0c7008fbfaca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d239c22857640e7b2ff0c7008fbfaca, State: Initialized, Role: FOLLOWER
I20260923 23:14:51.853379 511 catalog_manager.cc:4841] TS e9d6ab43fc0f491daef44c6e09be0e4a (127.24.227.195:39441): tablet 91ea62596e2a4ad4984dd3642df1912b (table test-table [id=e4769796168b4ac29069ed2157b09727]) successfully deleted
I20260923 23:14:51.853390 1039 consensus_queue.cc:260] T eb2d2d755d374109a0c172e42d4d691f P 9d239c22857640e7b2ff0c7008fbfaca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cd0dd070b8643bf966ede03f18a5de4" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 43785 } } peers { permanent_uuid: "9d239c22857640e7b2ff0c7008fbfaca" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 38277 } } peers { permanent_uuid: "e9d6ab43fc0f491daef44c6e09be0e4a" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39441 } }
I20260923 23:14:51.853485 1039 ts_tablet_manager.cc:1428] T eb2d2d755d374109a0c172e42d4d691f P 9d239c22857640e7b2ff0c7008fbfaca: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:51.854070 1059 ts_tablet_manager.cc:1918] T 91ea62596e2a4ad4984dd3642df1912b P 9d239c22857640e7b2ff0c7008fbfaca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 23:14:51.854138 1059 log.cc:1196] T 91ea62596e2a4ad4984dd3642df1912b P 9d239c22857640e7b2ff0c7008fbfaca: Deleting WAL directory at /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/minicluster-data/ts-1/wal/wals/91ea62596e2a4ad4984dd3642df1912b
I20260923 23:14:51.854425 1059 ts_tablet_manager.cc:1939] T 91ea62596e2a4ad4984dd3642df1912b P 9d239c22857640e7b2ff0c7008fbfaca: Deleting consensus metadata
I20260923 23:14:51.854795 513 catalog_manager.cc:4841] TS 9d239c22857640e7b2ff0c7008fbfaca (127.24.227.194:38277): tablet 91ea62596e2a4ad4984dd3642df1912b (table test-table [id=e4769796168b4ac29069ed2157b09727]) successfully deleted
W20260923 23:14:51.864526 871 tablet.cc:2369] T eb2d2d755d374109a0c172e42d4d691f P 9d239c22857640e7b2ff0c7008fbfaca: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 23:14:51.928599 1043 raft_consensus.cc:482] T eb2d2d755d374109a0c172e42d4d691f P 9d239c22857640e7b2ff0c7008fbfaca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 23:14:51.928707 1043 raft_consensus.cc:504] T eb2d2d755d374109a0c172e42d4d691f P 9d239c22857640e7b2ff0c7008fbfaca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cd0dd070b8643bf966ede03f18a5de4" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 43785 } } peers { permanent_uuid: "9d239c22857640e7b2ff0c7008fbfaca" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 38277 } } peers { permanent_uuid: "e9d6ab43fc0f491daef44c6e09be0e4a" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39441 } }
I20260923 23:14:51.928925 1043 leader_election.cc:288] T eb2d2d755d374109a0c172e42d4d691f P 9d239c22857640e7b2ff0c7008fbfaca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2cd0dd070b8643bf966ede03f18a5de4 (127.24.227.193:43785), e9d6ab43fc0f491daef44c6e09be0e4a (127.24.227.195:39441)
I20260923 23:14:51.929100 684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb2d2d755d374109a0c172e42d4d691f" candidate_uuid: "9d239c22857640e7b2ff0c7008fbfaca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cd0dd070b8643bf966ede03f18a5de4" is_pre_election: true
I20260923 23:14:51.929113 963 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb2d2d755d374109a0c172e42d4d691f" candidate_uuid: "9d239c22857640e7b2ff0c7008fbfaca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9d6ab43fc0f491daef44c6e09be0e4a" is_pre_election: true
I20260923 23:14:51.929222 684 raft_consensus.cc:2454] T eb2d2d755d374109a0c172e42d4d691f P 2cd0dd070b8643bf966ede03f18a5de4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d239c22857640e7b2ff0c7008fbfaca in term 0.
I20260923 23:14:51.929244 963 raft_consensus.cc:2454] T eb2d2d755d374109a0c172e42d4d691f P e9d6ab43fc0f491daef44c6e09be0e4a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d239c22857640e7b2ff0c7008fbfaca in term 0.
I20260923 23:14:51.929371 757 leader_election.cc:302] T eb2d2d755d374109a0c172e42d4d691f P 9d239c22857640e7b2ff0c7008fbfaca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2cd0dd070b8643bf966ede03f18a5de4, 9d239c22857640e7b2ff0c7008fbfaca, e9d6ab43fc0f491daef44c6e09be0e4a; no voters:
I20260923 23:14:51.929463 1043 raft_consensus.cc:2781] T eb2d2d755d374109a0c172e42d4d691f P 9d239c22857640e7b2ff0c7008fbfaca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 23:14:51.929493 1043 raft_consensus.cc:482] T eb2d2d755d374109a0c172e42d4d691f P 9d239c22857640e7b2ff0c7008fbfaca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 23:14:51.929520 1043 raft_consensus.cc:3037] T eb2d2d755d374109a0c172e42d4d691f P 9d239c22857640e7b2ff0c7008fbfaca [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:14:51.930220 1043 raft_consensus.cc:504] T eb2d2d755d374109a0c172e42d4d691f P 9d239c22857640e7b2ff0c7008fbfaca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cd0dd070b8643bf966ede03f18a5de4" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 43785 } } peers { permanent_uuid: "9d239c22857640e7b2ff0c7008fbfaca" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 38277 } } peers { permanent_uuid: "e9d6ab43fc0f491daef44c6e09be0e4a" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39441 } }
I20260923 23:14:51.930401 1043 leader_election.cc:288] T eb2d2d755d374109a0c172e42d4d691f P 9d239c22857640e7b2ff0c7008fbfaca [CANDIDATE]: Term 1 election: Requested vote from peers 2cd0dd070b8643bf966ede03f18a5de4 (127.24.227.193:43785), e9d6ab43fc0f491daef44c6e09be0e4a (127.24.227.195:39441)
I20260923 23:14:51.930517 684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb2d2d755d374109a0c172e42d4d691f" candidate_uuid: "9d239c22857640e7b2ff0c7008fbfaca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cd0dd070b8643bf966ede03f18a5de4"
I20260923 23:14:51.930545 963 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb2d2d755d374109a0c172e42d4d691f" candidate_uuid: "9d239c22857640e7b2ff0c7008fbfaca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9d6ab43fc0f491daef44c6e09be0e4a"
I20260923 23:14:51.930603 684 raft_consensus.cc:3037] T eb2d2d755d374109a0c172e42d4d691f P 2cd0dd070b8643bf966ede03f18a5de4 [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:14:51.930629 963 raft_consensus.cc:3037] T eb2d2d755d374109a0c172e42d4d691f P e9d6ab43fc0f491daef44c6e09be0e4a [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:14:51.931293 963 raft_consensus.cc:2454] T eb2d2d755d374109a0c172e42d4d691f P e9d6ab43fc0f491daef44c6e09be0e4a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d239c22857640e7b2ff0c7008fbfaca in term 1.
I20260923 23:14:51.931293 684 raft_consensus.cc:2454] T eb2d2d755d374109a0c172e42d4d691f P 2cd0dd070b8643bf966ede03f18a5de4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d239c22857640e7b2ff0c7008fbfaca in term 1.
I20260923 23:14:51.931450 755 leader_election.cc:302] T eb2d2d755d374109a0c172e42d4d691f P 9d239c22857640e7b2ff0c7008fbfaca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d239c22857640e7b2ff0c7008fbfaca, e9d6ab43fc0f491daef44c6e09be0e4a; no voters:
I20260923 23:14:51.931561 1043 raft_consensus.cc:2781] T eb2d2d755d374109a0c172e42d4d691f P 9d239c22857640e7b2ff0c7008fbfaca [term 1 FOLLOWER]: Leader election won for term 1
I20260923 23:14:51.931610 1043 raft_consensus.cc:686] T eb2d2d755d374109a0c172e42d4d691f P 9d239c22857640e7b2ff0c7008fbfaca [term 1 LEADER]: Becoming Leader. State: Replica: 9d239c22857640e7b2ff0c7008fbfaca, State: Running, Role: LEADER
I20260923 23:14:51.931658 1043 consensus_queue.cc:237] T eb2d2d755d374109a0c172e42d4d691f P 9d239c22857640e7b2ff0c7008fbfaca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cd0dd070b8643bf966ede03f18a5de4" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 43785 } } peers { permanent_uuid: "9d239c22857640e7b2ff0c7008fbfaca" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 38277 } } peers { permanent_uuid: "e9d6ab43fc0f491daef44c6e09be0e4a" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39441 } }
I20260923 23:14:51.932235 527 catalog_manager.cc:5495] T eb2d2d755d374109a0c172e42d4d691f P 9d239c22857640e7b2ff0c7008fbfaca reported cstate change: term changed from 0 to 1, leader changed from <none> to 9d239c22857640e7b2ff0c7008fbfaca (127.24.227.194). New cstate: current_term: 1 leader_uuid: "9d239c22857640e7b2ff0c7008fbfaca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cd0dd070b8643bf966ede03f18a5de4" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 43785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d239c22857640e7b2ff0c7008fbfaca" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 38277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9d6ab43fc0f491daef44c6e09be0e4a" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39441 } health_report { overall_health: UNKNOWN } } }
W20260923 23:14:51.953622 731 tablet.cc:2369] T eb2d2d755d374109a0c172e42d4d691f P 2cd0dd070b8643bf966ede03f18a5de4: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 23 23:14:51 dist-test-slave-754n krb5kdc[482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205291, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.227.254@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM
I20260923 23:14:51.978389 1069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:51.972517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:34607 (local address 127.0.0.1:58844)
0923 23:14:51.972667 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:51.972676 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:51.972748 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:51.972864 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:51.972864 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:51.973117 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:14:51.973162 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:51.973163 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:51.973717 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:51.973717 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:51.974190 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:51.974191 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:51.974459 (+ 268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:51.974460 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:51.974494 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:51.974992 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:14:51.974996 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:14:51.976228 (+ 1232us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:14:51.977480 (+ 1252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:51.977486 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:51.977487 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:51.977653 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:51.977810 (+ 157us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:51.977810 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:51.977811 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:51.977859 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:51.978111 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:14:51.978116 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:14:51.978154 (+ 38us) client_negotiation.cc:769] Sending connection context
0923 23:14:51.978206 (+ 52us) client_negotiation.cc:240] Negotiation successful
0923 23:14:51.978291 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":71,"threads_started":1}
I20260923 23:14:51.978459 596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:51.972584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58844 (local address 127.24.227.254:34607)
0923 23:14:51.972664 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:51.972666 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:51.972732 (+ 66us) server_negotiation.cc:407] Connection header received
0923 23:14:51.972793 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:51.972794 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:51.972802 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:51.972848 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:51.973217 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:51.973658 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:51.974252 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:51.974405 (+ 153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:51.974447 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:51.976306 (+ 1859us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:51.976307 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:51.976308 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:51.976313 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:51.977418 (+ 1105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:51.977710 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:51.977711 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:51.977711 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:51.977758 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:51.977905 (+ 147us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:51.977905 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:51.977905 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:51.978040 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:51.978099 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:51.978222 (+ 123us) server_negotiation.cc:299] Negotiation successful
0923 23:14:51.978336 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":50,"server-negotiator.queue_time_us":54}
I20260923 23:14:51.980463 527 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.24.227.254@KRBTEST.COM'} at 127.0.0.1:58844:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260923 23:14:51.980656 527 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 23:14:51.982043 804 tablet_service.cc:1467] Processing CreateTablet for tablet fde2bed12a8f45ae885f5b32bbcc8b8e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ef34469178c7400cbda6db627b2eba89]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260923 23:14:51.982056 664 tablet_service.cc:1467] Processing CreateTablet for tablet fde2bed12a8f45ae885f5b32bbcc8b8e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ef34469178c7400cbda6db627b2eba89]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260923 23:14:51.982050 944 tablet_service.cc:1467] Processing CreateTablet for tablet fde2bed12a8f45ae885f5b32bbcc8b8e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ef34469178c7400cbda6db627b2eba89]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260923 23:14:51.982173 804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fde2bed12a8f45ae885f5b32bbcc8b8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:14:51.982180 944 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fde2bed12a8f45ae885f5b32bbcc8b8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:14:51.982180 664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fde2bed12a8f45ae885f5b32bbcc8b8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:14:51.983529 1035 tablet_bootstrap.cc:492] T fde2bed12a8f45ae885f5b32bbcc8b8e P e9d6ab43fc0f491daef44c6e09be0e4a: Bootstrap starting.
I20260923 23:14:51.983552 1039 tablet_bootstrap.cc:492] T fde2bed12a8f45ae885f5b32bbcc8b8e P 9d239c22857640e7b2ff0c7008fbfaca: Bootstrap starting.
I20260923 23:14:51.983940 1039 tablet_bootstrap.cc:654] T fde2bed12a8f45ae885f5b32bbcc8b8e P 9d239c22857640e7b2ff0c7008fbfaca: Neither blocks nor log segments found. Creating new log.
I20260923 23:14:51.983947 1036 tablet_bootstrap.cc:492] T fde2bed12a8f45ae885f5b32bbcc8b8e P 2cd0dd070b8643bf966ede03f18a5de4: Bootstrap starting.
I20260923 23:14:51.984014 1035 tablet_bootstrap.cc:654] T fde2bed12a8f45ae885f5b32bbcc8b8e P e9d6ab43fc0f491daef44c6e09be0e4a: Neither blocks nor log segments found. Creating new log.
I20260923 23:14:51.984344 1036 tablet_bootstrap.cc:654] T fde2bed12a8f45ae885f5b32bbcc8b8e P 2cd0dd070b8643bf966ede03f18a5de4: Neither blocks nor log segments found. Creating new log.
I20260923 23:14:51.984503 1039 tablet_bootstrap.cc:492] T fde2bed12a8f45ae885f5b32bbcc8b8e P 9d239c22857640e7b2ff0c7008fbfaca: No bootstrap required, opened a new log
I20260923 23:14:51.984560 1039 ts_tablet_manager.cc:1397] T fde2bed12a8f45ae885f5b32bbcc8b8e P 9d239c22857640e7b2ff0c7008fbfaca: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260923 23:14:51.984691 1039 raft_consensus.cc:348] T fde2bed12a8f45ae885f5b32bbcc8b8e P 9d239c22857640e7b2ff0c7008fbfaca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cd0dd070b8643bf966ede03f18a5de4" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 43785 } } peers { permanent_uuid: "9d239c22857640e7b2ff0c7008fbfaca" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 38277 } } peers { permanent_uuid: "e9d6ab43fc0f491daef44c6e09be0e4a" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39441 } }
I20260923 23:14:51.984771 1039 raft_consensus.cc:374] T fde2bed12a8f45ae885f5b32bbcc8b8e P 9d239c22857640e7b2ff0c7008fbfaca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:14:51.984794 1039 raft_consensus.cc:729] T fde2bed12a8f45ae885f5b32bbcc8b8e P 9d239c22857640e7b2ff0c7008fbfaca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d239c22857640e7b2ff0c7008fbfaca, State: Initialized, Role: FOLLOWER
I20260923 23:14:51.984812 1035 tablet_bootstrap.cc:492] T fde2bed12a8f45ae885f5b32bbcc8b8e P e9d6ab43fc0f491daef44c6e09be0e4a: No bootstrap required, opened a new log
I20260923 23:14:51.984855 1035 ts_tablet_manager.cc:1397] T fde2bed12a8f45ae885f5b32bbcc8b8e P e9d6ab43fc0f491daef44c6e09be0e4a: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260923 23:14:51.984844 1039 consensus_queue.cc:260] T fde2bed12a8f45ae885f5b32bbcc8b8e P 9d239c22857640e7b2ff0c7008fbfaca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cd0dd070b8643bf966ede03f18a5de4" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 43785 } } peers { permanent_uuid: "9d239c22857640e7b2ff0c7008fbfaca" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 38277 } } peers { permanent_uuid: "e9d6ab43fc0f491daef44c6e09be0e4a" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39441 } }
I20260923 23:14:51.984964 1039 ts_tablet_manager.cc:1428] T fde2bed12a8f45ae885f5b32bbcc8b8e P 9d239c22857640e7b2ff0c7008fbfaca: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260923 23:14:51.985011 1035 raft_consensus.cc:348] T fde2bed12a8f45ae885f5b32bbcc8b8e P e9d6ab43fc0f491daef44c6e09be0e4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cd0dd070b8643bf966ede03f18a5de4" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 43785 } } peers { permanent_uuid: "9d239c22857640e7b2ff0c7008fbfaca" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 38277 } } peers { permanent_uuid: "e9d6ab43fc0f491daef44c6e09be0e4a" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39441 } }
I20260923 23:14:51.985060 1036 tablet_bootstrap.cc:492] T fde2bed12a8f45ae885f5b32bbcc8b8e P 2cd0dd070b8643bf966ede03f18a5de4: No bootstrap required, opened a new log
I20260923 23:14:51.985074 1035 raft_consensus.cc:374] T fde2bed12a8f45ae885f5b32bbcc8b8e P e9d6ab43fc0f491daef44c6e09be0e4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:14:51.985093 1035 raft_consensus.cc:729] T fde2bed12a8f45ae885f5b32bbcc8b8e P e9d6ab43fc0f491daef44c6e09be0e4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9d6ab43fc0f491daef44c6e09be0e4a, State: Initialized, Role: FOLLOWER
I20260923 23:14:51.985052 1043 tablet_replica.cc:439] T fde2bed12a8f45ae885f5b32bbcc8b8e P 9d239c22857640e7b2ff0c7008fbfaca: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cd0dd070b8643bf966ede03f18a5de4" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 43785 } } peers { permanent_uuid: "9d239c22857640e7b2ff0c7008fbfaca" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 38277 } } peers { permanent_uuid: "e9d6ab43fc0f491daef44c6e09be0e4a" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39441 } } }
I20260923 23:14:51.985097 1036 ts_tablet_manager.cc:1397] T fde2bed12a8f45ae885f5b32bbcc8b8e P 2cd0dd070b8643bf966ede03f18a5de4: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260923 23:14:51.985121 1043 tablet_replica.cc:442] T fde2bed12a8f45ae885f5b32bbcc8b8e P 9d239c22857640e7b2ff0c7008fbfaca: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 23:14:51.985141 1035 consensus_queue.cc:260] T fde2bed12a8f45ae885f5b32bbcc8b8e P e9d6ab43fc0f491daef44c6e09be0e4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cd0dd070b8643bf966ede03f18a5de4" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 43785 } } peers { permanent_uuid: "9d239c22857640e7b2ff0c7008fbfaca" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 38277 } } peers { permanent_uuid: "e9d6ab43fc0f491daef44c6e09be0e4a" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39441 } }
I20260923 23:14:51.985255 1035 ts_tablet_manager.cc:1428] T fde2bed12a8f45ae885f5b32bbcc8b8e P e9d6ab43fc0f491daef44c6e09be0e4a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:51.985250 1036 raft_consensus.cc:348] T fde2bed12a8f45ae885f5b32bbcc8b8e P 2cd0dd070b8643bf966ede03f18a5de4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cd0dd070b8643bf966ede03f18a5de4" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 43785 } } peers { permanent_uuid: "9d239c22857640e7b2ff0c7008fbfaca" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 38277 } } peers { permanent_uuid: "e9d6ab43fc0f491daef44c6e09be0e4a" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39441 } }
I20260923 23:14:51.985316 1036 raft_consensus.cc:374] T fde2bed12a8f45ae885f5b32bbcc8b8e P 2cd0dd070b8643bf966ede03f18a5de4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:14:51.985337 1036 raft_consensus.cc:729] T fde2bed12a8f45ae885f5b32bbcc8b8e P 2cd0dd070b8643bf966ede03f18a5de4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cd0dd070b8643bf966ede03f18a5de4, State: Initialized, Role: FOLLOWER
I20260923 23:14:51.985297 1041 tablet_replica.cc:439] T fde2bed12a8f45ae885f5b32bbcc8b8e P e9d6ab43fc0f491daef44c6e09be0e4a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cd0dd070b8643bf966ede03f18a5de4" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 43785 } } peers { permanent_uuid: "9d239c22857640e7b2ff0c7008fbfaca" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 38277 } } peers { permanent_uuid: "e9d6ab43fc0f491daef44c6e09be0e4a" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39441 } } }
I20260923 23:14:51.985385 1041 tablet_replica.cc:442] T fde2bed12a8f45ae885f5b32bbcc8b8e P e9d6ab43fc0f491daef44c6e09be0e4a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 23:14:51.985378 1036 consensus_queue.cc:260] T fde2bed12a8f45ae885f5b32bbcc8b8e P 2cd0dd070b8643bf966ede03f18a5de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cd0dd070b8643bf966ede03f18a5de4" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 43785 } } peers { permanent_uuid: "9d239c22857640e7b2ff0c7008fbfaca" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 38277 } } peers { permanent_uuid: "e9d6ab43fc0f491daef44c6e09be0e4a" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39441 } }
I20260923 23:14:51.985478 1036 ts_tablet_manager.cc:1428] T fde2bed12a8f45ae885f5b32bbcc8b8e P 2cd0dd070b8643bf966ede03f18a5de4: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:51.985580 1042 tablet_replica.cc:439] T fde2bed12a8f45ae885f5b32bbcc8b8e P 2cd0dd070b8643bf966ede03f18a5de4: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cd0dd070b8643bf966ede03f18a5de4" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 43785 } } peers { permanent_uuid: "9d239c22857640e7b2ff0c7008fbfaca" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 38277 } } peers { permanent_uuid: "e9d6ab43fc0f491daef44c6e09be0e4a" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39441 } } }
I20260923 23:14:51.985626 1042 tablet_replica.cc:442] T fde2bed12a8f45ae885f5b32bbcc8b8e P 2cd0dd070b8643bf966ede03f18a5de4: This TxnStatusTablet replica's current role is: FOLLOWER
W20260923 23:14:52.011909 1011 tablet.cc:2369] T eb2d2d755d374109a0c172e42d4d691f P e9d6ab43fc0f491daef44c6e09be0e4a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 23:14:52.032862 1042 raft_consensus.cc:482] T fde2bed12a8f45ae885f5b32bbcc8b8e P 2cd0dd070b8643bf966ede03f18a5de4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 23:14:52.032941 1042 raft_consensus.cc:504] T fde2bed12a8f45ae885f5b32bbcc8b8e P 2cd0dd070b8643bf966ede03f18a5de4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cd0dd070b8643bf966ede03f18a5de4" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 43785 } } peers { permanent_uuid: "9d239c22857640e7b2ff0c7008fbfaca" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 38277 } } peers { permanent_uuid: "e9d6ab43fc0f491daef44c6e09be0e4a" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39441 } }
I20260923 23:14:52.033282 1042 leader_election.cc:288] T fde2bed12a8f45ae885f5b32bbcc8b8e P 2cd0dd070b8643bf966ede03f18a5de4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9d239c22857640e7b2ff0c7008fbfaca (127.24.227.194:38277), e9d6ab43fc0f491daef44c6e09be0e4a (127.24.227.195:39441)
I20260923 23:14:52.036145 1034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:52.033512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.193:48883 (local address 127.24.227.195:39441)
0923 23:14:52.033553 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:52.033555 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:52.033615 (+ 60us) server_negotiation.cc:407] Connection header received
0923 23:14:52.033664 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:52.033665 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:52.033673 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:52.033772 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:14:52.033912 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:52.034386 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:52.035439 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:52.035780 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:52.035888 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:52.035897 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:52.036039 (+ 142us) server_negotiation.cc:299] Negotiation successful
0923 23:14:52.036086 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260923 23:14:52.036162 1074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:52.033416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.195:39441 (local address 127.24.227.193:48883)
0923 23:14:52.033543 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:52.033553 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:52.033612 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:52.033778 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:52.033779 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:52.033781 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:14:52.033845 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:52.033847 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:52.034466 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:52.034467 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:52.035357 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:52.035359 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:52.035888 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:52.035889 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:52.035952 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:52.035957 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 23:14:52.036033 (+ 76us) client_negotiation.cc:240] Negotiation successful
0923 23:14:52.036071 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"mutex_wait_us":3,"thread_start_us":67,"threads_started":1}
I20260923 23:14:52.036237 1031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:52.033479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.193:43233 (local address 127.24.227.194:38277)
0923 23:14:52.033527 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:52.033529 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:52.033596 (+ 67us) server_negotiation.cc:407] Connection header received
0923 23:14:52.033697 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:52.033698 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:52.033729 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:52.033822 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:14:52.033960 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:52.034397 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:52.035421 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:52.035820 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:52.035951 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:52.035959 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:52.036109 (+ 150us) server_negotiation.cc:299] Negotiation successful
0923 23:14:52.036146 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 23:14:52.036298 1073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:52.033416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.194:38277 (local address 127.24.227.193:43233)
0923 23:14:52.033529 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:52.033538 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:52.033628 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:52.033825 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:52.033826 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:52.033828 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:14:52.033884 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:52.033886 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:52.034480 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:52.034481 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:52.035336 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:52.035338 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:52.035952 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:52.035953 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:52.036016 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:52.036019 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 23:14:52.036090 (+ 71us) client_negotiation.cc:240] Negotiation successful
0923 23:14:52.036129 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"mutex_wait_us":1,"thread_start_us":60,"threads_started":1}
I20260923 23:14:52.036443 963 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fde2bed12a8f45ae885f5b32bbcc8b8e" candidate_uuid: "2cd0dd070b8643bf966ede03f18a5de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9d6ab43fc0f491daef44c6e09be0e4a" is_pre_election: true
I20260923 23:14:52.036515 963 raft_consensus.cc:2454] T fde2bed12a8f45ae885f5b32bbcc8b8e P e9d6ab43fc0f491daef44c6e09be0e4a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2cd0dd070b8643bf966ede03f18a5de4 in term 0.
I20260923 23:14:52.036722 615 leader_election.cc:302] T fde2bed12a8f45ae885f5b32bbcc8b8e P 2cd0dd070b8643bf966ede03f18a5de4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cd0dd070b8643bf966ede03f18a5de4, e9d6ab43fc0f491daef44c6e09be0e4a; no voters:
I20260923 23:14:52.036710 824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fde2bed12a8f45ae885f5b32bbcc8b8e" candidate_uuid: "2cd0dd070b8643bf966ede03f18a5de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d239c22857640e7b2ff0c7008fbfaca" is_pre_election: true
I20260923 23:14:52.036811 824 raft_consensus.cc:2454] T fde2bed12a8f45ae885f5b32bbcc8b8e P 9d239c22857640e7b2ff0c7008fbfaca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2cd0dd070b8643bf966ede03f18a5de4 in term 0.
I20260923 23:14:52.036969 1042 raft_consensus.cc:2781] T fde2bed12a8f45ae885f5b32bbcc8b8e P 2cd0dd070b8643bf966ede03f18a5de4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 23:14:52.037006 1042 raft_consensus.cc:482] T fde2bed12a8f45ae885f5b32bbcc8b8e P 2cd0dd070b8643bf966ede03f18a5de4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 23:14:52.037020 1042 raft_consensus.cc:3037] T fde2bed12a8f45ae885f5b32bbcc8b8e P 2cd0dd070b8643bf966ede03f18a5de4 [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:14:52.037752 1042 raft_consensus.cc:504] T fde2bed12a8f45ae885f5b32bbcc8b8e P 2cd0dd070b8643bf966ede03f18a5de4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cd0dd070b8643bf966ede03f18a5de4" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 43785 } } peers { permanent_uuid: "9d239c22857640e7b2ff0c7008fbfaca" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 38277 } } peers { permanent_uuid: "e9d6ab43fc0f491daef44c6e09be0e4a" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39441 } }
I20260923 23:14:52.037889 1042 leader_election.cc:288] T fde2bed12a8f45ae885f5b32bbcc8b8e P 2cd0dd070b8643bf966ede03f18a5de4 [CANDIDATE]: Term 1 election: Requested vote from peers 9d239c22857640e7b2ff0c7008fbfaca (127.24.227.194:38277), e9d6ab43fc0f491daef44c6e09be0e4a (127.24.227.195:39441)
I20260923 23:14:52.038013 824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fde2bed12a8f45ae885f5b32bbcc8b8e" candidate_uuid: "2cd0dd070b8643bf966ede03f18a5de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d239c22857640e7b2ff0c7008fbfaca"
I20260923 23:14:52.038020 963 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fde2bed12a8f45ae885f5b32bbcc8b8e" candidate_uuid: "2cd0dd070b8643bf966ede03f18a5de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9d6ab43fc0f491daef44c6e09be0e4a"
I20260923 23:14:52.038084 824 raft_consensus.cc:3037] T fde2bed12a8f45ae885f5b32bbcc8b8e P 9d239c22857640e7b2ff0c7008fbfaca [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:14:52.038100 963 raft_consensus.cc:3037] T fde2bed12a8f45ae885f5b32bbcc8b8e P e9d6ab43fc0f491daef44c6e09be0e4a [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:14:52.038738 824 raft_consensus.cc:2454] T fde2bed12a8f45ae885f5b32bbcc8b8e P 9d239c22857640e7b2ff0c7008fbfaca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2cd0dd070b8643bf966ede03f18a5de4 in term 1.
I20260923 23:14:52.038753 963 raft_consensus.cc:2454] T fde2bed12a8f45ae885f5b32bbcc8b8e P e9d6ab43fc0f491daef44c6e09be0e4a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2cd0dd070b8643bf966ede03f18a5de4 in term 1.
I20260923 23:14:52.038908 617 leader_election.cc:302] T fde2bed12a8f45ae885f5b32bbcc8b8e P 2cd0dd070b8643bf966ede03f18a5de4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cd0dd070b8643bf966ede03f18a5de4, 9d239c22857640e7b2ff0c7008fbfaca; no voters:
I20260923 23:14:52.039013 1042 raft_consensus.cc:2781] T fde2bed12a8f45ae885f5b32bbcc8b8e P 2cd0dd070b8643bf966ede03f18a5de4 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 23:14:52.039153 1042 raft_consensus.cc:686] T fde2bed12a8f45ae885f5b32bbcc8b8e P 2cd0dd070b8643bf966ede03f18a5de4 [term 1 LEADER]: Becoming Leader. State: Replica: 2cd0dd070b8643bf966ede03f18a5de4, State: Running, Role: LEADER
I20260923 23:14:52.039219 1042 consensus_queue.cc:237] T fde2bed12a8f45ae885f5b32bbcc8b8e P 2cd0dd070b8643bf966ede03f18a5de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cd0dd070b8643bf966ede03f18a5de4" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 43785 } } peers { permanent_uuid: "9d239c22857640e7b2ff0c7008fbfaca" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 38277 } } peers { permanent_uuid: "e9d6ab43fc0f491daef44c6e09be0e4a" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39441 } }
I20260923 23:14:52.039597 1075 tablet_replica.cc:439] T fde2bed12a8f45ae885f5b32bbcc8b8e P 2cd0dd070b8643bf966ede03f18a5de4: TxnStatusTablet state changed. Reason: New leader 2cd0dd070b8643bf966ede03f18a5de4. Latest consensus state: current_term: 1 leader_uuid: "2cd0dd070b8643bf966ede03f18a5de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cd0dd070b8643bf966ede03f18a5de4" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 43785 } } peers { permanent_uuid: "9d239c22857640e7b2ff0c7008fbfaca" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 38277 } } peers { permanent_uuid: "e9d6ab43fc0f491daef44c6e09be0e4a" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39441 } } }
I20260923 23:14:52.039691 1075 tablet_replica.cc:442] T fde2bed12a8f45ae885f5b32bbcc8b8e P 2cd0dd070b8643bf966ede03f18a5de4: This TxnStatusTablet replica's current role is: LEADER
I20260923 23:14:52.039881 1077 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260923 23:14:52.040010 1077 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260923 23:14:52.039964 527 catalog_manager.cc:5495] T fde2bed12a8f45ae885f5b32bbcc8b8e P 2cd0dd070b8643bf966ede03f18a5de4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2cd0dd070b8643bf966ede03f18a5de4 (127.24.227.193). New cstate: current_term: 1 leader_uuid: "2cd0dd070b8643bf966ede03f18a5de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cd0dd070b8643bf966ede03f18a5de4" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 43785 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d239c22857640e7b2ff0c7008fbfaca" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 38277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9d6ab43fc0f491daef44c6e09be0e4a" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39441 } health_report { overall_health: UNKNOWN } } }
I20260923 23:14:52.059733 1069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:52.057318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.193:43785 (local address 127.0.0.1:57866)
0923 23:14:52.057388 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:52.057400 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:52.057461 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:52.057611 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:52.057612 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:52.057613 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0923 23:14:52.057656 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:52.057659 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:52.058268 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:52.058270 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:52.058840 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:52.058841 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:52.059198 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:52.059199 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:52.059271 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:52.059292 (+ 21us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0923 23:14:52.059553 (+ 261us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0923 23:14:52.059561 (+ 8us) client_negotiation.cc:769] Sending connection context
0923 23:14:52.059619 (+ 58us) client_negotiation.cc:240] Negotiation successful
0923 23:14:52.059666 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260923 23:14:52.059716 1033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:52.057369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57866 (local address 127.24.227.193:43785)
0923 23:14:52.057418 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:52.057421 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:52.057443 (+ 22us) server_negotiation.cc:407] Connection header received
0923 23:14:52.057536 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:52.057537 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:52.057553 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:52.057598 (+ 45us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 23:14:52.057728 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:52.058184 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:52.058922 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:52.059125 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:52.059199 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:52.059390 (+ 191us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 23:14:52.059486 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 23:14:52.059566 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:52.059632 (+ 66us) server_negotiation.cc:299] Negotiation successful
0923 23:14:52.059652 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260923 23:14:52.060799 824 raft_consensus.cc:1261] T fde2bed12a8f45ae885f5b32bbcc8b8e P 9d239c22857640e7b2ff0c7008fbfaca [term 1 FOLLOWER]: Refusing update from remote peer 2cd0dd070b8643bf966ede03f18a5de4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 23:14:52.060796 963 raft_consensus.cc:1261] T fde2bed12a8f45ae885f5b32bbcc8b8e P e9d6ab43fc0f491daef44c6e09be0e4a [term 1 FOLLOWER]: Refusing update from remote peer 2cd0dd070b8643bf966ede03f18a5de4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 23:14:52.061039 1042 consensus_queue.cc:1035] T fde2bed12a8f45ae885f5b32bbcc8b8e P 2cd0dd070b8643bf966ede03f18a5de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9d6ab43fc0f491daef44c6e09be0e4a" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39441 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 23:14:52.061209 1075 consensus_queue.cc:1035] T fde2bed12a8f45ae885f5b32bbcc8b8e P 2cd0dd070b8643bf966ede03f18a5de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d239c22857640e7b2ff0c7008fbfaca" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 38277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 23:14:52.061580 1041 tablet_replica.cc:439] T fde2bed12a8f45ae885f5b32bbcc8b8e P e9d6ab43fc0f491daef44c6e09be0e4a: TxnStatusTablet state changed. Reason: New leader 2cd0dd070b8643bf966ede03f18a5de4. Latest consensus state: current_term: 1 leader_uuid: "2cd0dd070b8643bf966ede03f18a5de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cd0dd070b8643bf966ede03f18a5de4" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 43785 } } peers { permanent_uuid: "9d239c22857640e7b2ff0c7008fbfaca" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 38277 } } peers { permanent_uuid: "e9d6ab43fc0f491daef44c6e09be0e4a" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39441 } } }
I20260923 23:14:52.061667 1041 tablet_replica.cc:442] T fde2bed12a8f45ae885f5b32bbcc8b8e P e9d6ab43fc0f491daef44c6e09be0e4a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 23:14:52.062479 1042 tablet_replica.cc:439] T fde2bed12a8f45ae885f5b32bbcc8b8e P 2cd0dd070b8643bf966ede03f18a5de4: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2cd0dd070b8643bf966ede03f18a5de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cd0dd070b8643bf966ede03f18a5de4" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 43785 } } peers { permanent_uuid: "9d239c22857640e7b2ff0c7008fbfaca" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 38277 } } peers { permanent_uuid: "e9d6ab43fc0f491daef44c6e09be0e4a" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39441 } } }
I20260923 23:14:52.062557 1042 tablet_replica.cc:442] T fde2bed12a8f45ae885f5b32bbcc8b8e P 2cd0dd070b8643bf966ede03f18a5de4: This TxnStatusTablet replica's current role is: LEADER
I20260923 23:14:52.062516 1043 tablet_replica.cc:439] T fde2bed12a8f45ae885f5b32bbcc8b8e P 9d239c22857640e7b2ff0c7008fbfaca: TxnStatusTablet state changed. Reason: New leader 2cd0dd070b8643bf966ede03f18a5de4. Latest consensus state: current_term: 1 leader_uuid: "2cd0dd070b8643bf966ede03f18a5de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cd0dd070b8643bf966ede03f18a5de4" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 43785 } } peers { permanent_uuid: "9d239c22857640e7b2ff0c7008fbfaca" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 38277 } } peers { permanent_uuid: "e9d6ab43fc0f491daef44c6e09be0e4a" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39441 } } }
I20260923 23:14:52.062585 1043 tablet_replica.cc:442] T fde2bed12a8f45ae885f5b32bbcc8b8e P 9d239c22857640e7b2ff0c7008fbfaca: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 23:14:52.063431 1041 tablet_replica.cc:439] T fde2bed12a8f45ae885f5b32bbcc8b8e P e9d6ab43fc0f491daef44c6e09be0e4a: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2cd0dd070b8643bf966ede03f18a5de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cd0dd070b8643bf966ede03f18a5de4" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 43785 } } peers { permanent_uuid: "9d239c22857640e7b2ff0c7008fbfaca" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 38277 } } peers { permanent_uuid: "e9d6ab43fc0f491daef44c6e09be0e4a" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39441 } } }
I20260923 23:14:52.063515 1041 tablet_replica.cc:442] T fde2bed12a8f45ae885f5b32bbcc8b8e P e9d6ab43fc0f491daef44c6e09be0e4a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 23:14:52.064332 1042 tablet_replica.cc:439] T fde2bed12a8f45ae885f5b32bbcc8b8e P 2cd0dd070b8643bf966ede03f18a5de4: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2cd0dd070b8643bf966ede03f18a5de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cd0dd070b8643bf966ede03f18a5de4" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 43785 } } peers { permanent_uuid: "9d239c22857640e7b2ff0c7008fbfaca" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 38277 } } peers { permanent_uuid: "e9d6ab43fc0f491daef44c6e09be0e4a" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39441 } } }
I20260923 23:14:52.064406 1042 tablet_replica.cc:442] T fde2bed12a8f45ae885f5b32bbcc8b8e P 2cd0dd070b8643bf966ede03f18a5de4: This TxnStatusTablet replica's current role is: LEADER
I20260923 23:14:52.064936 1043 tablet_replica.cc:439] T fde2bed12a8f45ae885f5b32bbcc8b8e P 9d239c22857640e7b2ff0c7008fbfaca: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2cd0dd070b8643bf966ede03f18a5de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cd0dd070b8643bf966ede03f18a5de4" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 43785 } } peers { permanent_uuid: "9d239c22857640e7b2ff0c7008fbfaca" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 38277 } } peers { permanent_uuid: "e9d6ab43fc0f491daef44c6e09be0e4a" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39441 } } }
I20260923 23:14:52.065018 1043 tablet_replica.cc:442] T fde2bed12a8f45ae885f5b32bbcc8b8e P 9d239c22857640e7b2ff0c7008fbfaca: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 23:14:52.068188 1033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:52.066065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.194:39723 (local address 127.24.227.193:43785)
0923 23:14:52.066108 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:52.066110 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:52.066119 (+ 9us) server_negotiation.cc:407] Connection header received
0923 23:14:52.066173 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:52.066174 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:52.066183 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:52.066260 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 23:14:52.066360 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:52.066809 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:52.067567 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:52.067755 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:52.067820 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:52.067964 (+ 144us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 23:14:52.067991 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 23:14:52.068053 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:52.068099 (+ 46us) server_negotiation.cc:299] Negotiation successful
0923 23:14:52.068113 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260923 23:14:52.068226 876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:52.066006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.193:43785 (local address 127.24.227.194:39723)
0923 23:14:52.066072 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:52.066083 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:52.066124 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:52.066264 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:52.066264 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:52.066265 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0923 23:14:52.066307 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:52.066308 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:52.066886 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:52.066887 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:52.067462 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:52.067463 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:52.067820 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:52.067821 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:52.067891 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:52.067904 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0923 23:14:52.068053 (+ 149us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0923 23:14:52.068055 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 23:14:52.068097 (+ 42us) client_negotiation.cc:240] Negotiation successful
0923 23:14:52.068159 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260923 23:14:52.070116 684 raft_consensus.cc:1261] T eb2d2d755d374109a0c172e42d4d691f P 2cd0dd070b8643bf966ede03f18a5de4 [term 1 FOLLOWER]: Refusing update from remote peer 9d239c22857640e7b2ff0c7008fbfaca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 23:14:52.070118 963 raft_consensus.cc:1261] T eb2d2d755d374109a0c172e42d4d691f P e9d6ab43fc0f491daef44c6e09be0e4a [term 1 FOLLOWER]: Refusing update from remote peer 9d239c22857640e7b2ff0c7008fbfaca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 23:14:52.070273 1043 consensus_queue.cc:1035] T eb2d2d755d374109a0c172e42d4d691f P 9d239c22857640e7b2ff0c7008fbfaca [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9d6ab43fc0f491daef44c6e09be0e4a" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39441 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 23:14:52.070369 1045 consensus_queue.cc:1035] T eb2d2d755d374109a0c172e42d4d691f P 9d239c22857640e7b2ff0c7008fbfaca [LEADER]: Connected to new peer: Peer: permanent_uuid: "2cd0dd070b8643bf966ede03f18a5de4" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 43785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 23:14:52.071789 1048 mvcc.cc:205] Tried to move back new op lower bound from 7332680876317265920 to 7332680875752120320. Current Snapshot: MvccSnapshot[applied={T|T < 7332680876317265920}]
I20260923 23:14:52.078415 1031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:52.075956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.193:52619 (local address 127.24.227.194:38277)
0923 23:14:52.075986 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:52.075988 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:52.076076 (+ 88us) server_negotiation.cc:407] Connection header received
0923 23:14:52.076135 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:52.076135 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:52.076142 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:52.076177 (+ 35us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 23:14:52.076339 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:52.076760 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:52.077832 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:52.077998 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:52.078043 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:52.078203 (+ 160us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 23:14:52.078267 (+ 64us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 23:14:52.078305 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:52.078363 (+ 58us) server_negotiation.cc:299] Negotiation successful
0923 23:14:52.078378 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260923 23:14:52.078480 1088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:52.075855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.194:38277 (local address 127.24.227.193:52619)
0923 23:14:52.076026 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:52.076035 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:52.076086 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:52.076205 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:52.076205 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:52.076209 (+ 4us) client_negotiation.cc:189] Negotiated authn=TOKEN
0923 23:14:52.076288 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:52.076289 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:52.076828 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:52.076829 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:52.077758 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:52.077760 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:52.078061 (+ 301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:52.078062 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:52.078121 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:52.078134 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0923 23:14:52.078316 (+ 182us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0923 23:14:52.078318 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 23:14:52.078361 (+ 43us) client_negotiation.cc:240] Negotiation successful
0923 23:14:52.078396 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":103,"threads_started":1}
I20260923 23:14:52.086908 526 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58832:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 23:14:52.086985 526 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58832:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 23:14:52.087658 526 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 607f68c4a2fb4f858b83bbc07edd2bdb: Sending DeleteTablet for 3 replicas of tablet eb2d2d755d374109a0c172e42d4d691f
I20260923 23:14:52.087983 664 tablet_service.cc:1514] Processing DeleteTablet for tablet eb2d2d755d374109a0c172e42d4d691f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 23:14:52 UTC) from {username='oryx', principal='oryx/127.24.227.254@KRBTEST.COM'} at 127.0.0.1:57858
I20260923 23:14:52.088001 944 tablet_service.cc:1514] Processing DeleteTablet for tablet eb2d2d755d374109a0c172e42d4d691f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 23:14:52 UTC) from {username='oryx', principal='oryx/127.24.227.254@KRBTEST.COM'} at 127.0.0.1:43294
I20260923 23:14:52.088070 1058 tablet_replica.cc:330] T eb2d2d755d374109a0c172e42d4d691f P 2cd0dd070b8643bf966ede03f18a5de4: stopping tablet replica
I20260923 23:14:52.088114 1058 raft_consensus.cc:2229] T eb2d2d755d374109a0c172e42d4d691f P 2cd0dd070b8643bf966ede03f18a5de4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 23:14:52.088143 1058 raft_consensus.cc:2258] T eb2d2d755d374109a0c172e42d4d691f P 2cd0dd070b8643bf966ede03f18a5de4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 23:14:52.088203 1057 tablet_replica.cc:330] T eb2d2d755d374109a0c172e42d4d691f P e9d6ab43fc0f491daef44c6e09be0e4a: stopping tablet replica
I20260923 23:14:52.088249 1057 raft_consensus.cc:2229] T eb2d2d755d374109a0c172e42d4d691f P e9d6ab43fc0f491daef44c6e09be0e4a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 23:14:52.088290 1057 raft_consensus.cc:2258] T eb2d2d755d374109a0c172e42d4d691f P e9d6ab43fc0f491daef44c6e09be0e4a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 23:14:52.088584 1058 ts_tablet_manager.cc:1905] T eb2d2d755d374109a0c172e42d4d691f P 2cd0dd070b8643bf966ede03f18a5de4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 23:14:52.089840 804 tablet_service.cc:1514] Processing DeleteTablet for tablet eb2d2d755d374109a0c172e42d4d691f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 23:14:52 UTC) from {username='oryx', principal='oryx/127.24.227.254@KRBTEST.COM'} at 127.0.0.1:35736
I20260923 23:14:52.089922 1059 tablet_replica.cc:330] T eb2d2d755d374109a0c172e42d4d691f P 9d239c22857640e7b2ff0c7008fbfaca: stopping tablet replica
I20260923 23:14:52.089986 1059 raft_consensus.cc:2229] T eb2d2d755d374109a0c172e42d4d691f P 9d239c22857640e7b2ff0c7008fbfaca [term 1 LEADER]: Raft consensus shutting down.
I20260923 23:14:52.090047 1059 raft_consensus.cc:2258] T eb2d2d755d374109a0c172e42d4d691f P 9d239c22857640e7b2ff0c7008fbfaca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 23:14:52.090423 1058 ts_tablet_manager.cc:1918] T eb2d2d755d374109a0c172e42d4d691f P 2cd0dd070b8643bf966ede03f18a5de4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 23:14:52.090425 1059 ts_tablet_manager.cc:1905] T eb2d2d755d374109a0c172e42d4d691f P 9d239c22857640e7b2ff0c7008fbfaca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 23:14:52.090485 1058 log.cc:1196] T eb2d2d755d374109a0c172e42d4d691f P 2cd0dd070b8643bf966ede03f18a5de4: Deleting WAL directory at /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/minicluster-data/ts-0/wal/wals/eb2d2d755d374109a0c172e42d4d691f
I20260923 23:14:52.090577 1057 ts_tablet_manager.cc:1905] T eb2d2d755d374109a0c172e42d4d691f P e9d6ab43fc0f491daef44c6e09be0e4a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 23:14:52.090708 1058 ts_tablet_manager.cc:1939] T eb2d2d755d374109a0c172e42d4d691f P 2cd0dd070b8643bf966ede03f18a5de4: Deleting consensus metadata
I20260923 23:14:52.091075 513 catalog_manager.cc:4841] TS 2cd0dd070b8643bf966ede03f18a5de4 (127.24.227.193:43785): tablet eb2d2d755d374109a0c172e42d4d691f (table test-table [id=65cbc566eb29437daec5273774504e55]) successfully deleted
I20260923 23:14:52.092280 1059 ts_tablet_manager.cc:1918] T eb2d2d755d374109a0c172e42d4d691f P 9d239c22857640e7b2ff0c7008fbfaca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 23:14:52.092346 1059 log.cc:1196] T eb2d2d755d374109a0c172e42d4d691f P 9d239c22857640e7b2ff0c7008fbfaca: Deleting WAL directory at /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/minicluster-data/ts-1/wal/wals/eb2d2d755d374109a0c172e42d4d691f
I20260923 23:14:52.092579 1059 ts_tablet_manager.cc:1939] T eb2d2d755d374109a0c172e42d4d691f P 9d239c22857640e7b2ff0c7008fbfaca: Deleting consensus metadata
I20260923 23:14:52.092890 1057 ts_tablet_manager.cc:1918] T eb2d2d755d374109a0c172e42d4d691f P e9d6ab43fc0f491daef44c6e09be0e4a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 23:14:52.092967 1057 log.cc:1196] T eb2d2d755d374109a0c172e42d4d691f P e9d6ab43fc0f491daef44c6e09be0e4a: Deleting WAL directory at /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/minicluster-data/ts-2/wal/wals/eb2d2d755d374109a0c172e42d4d691f
I20260923 23:14:52.092989 513 catalog_manager.cc:4841] TS 9d239c22857640e7b2ff0c7008fbfaca (127.24.227.194:38277): tablet eb2d2d755d374109a0c172e42d4d691f (table test-table [id=65cbc566eb29437daec5273774504e55]) successfully deleted
I20260923 23:14:52.093164 1057 ts_tablet_manager.cc:1939] T eb2d2d755d374109a0c172e42d4d691f P e9d6ab43fc0f491daef44c6e09be0e4a: Deleting consensus metadata
I20260923 23:14:52.093473 511 catalog_manager.cc:4841] TS e9d6ab43fc0f491daef44c6e09be0e4a (127.24.227.195:39441): tablet eb2d2d755d374109a0c172e42d4d691f (table test-table [id=65cbc566eb29437daec5273774504e55]) successfully deleted
Sep 23 23:14:52 dist-test-slave-754n krb5kdc[482](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205292, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 23 23:14:52 dist-test-slave-754n krb5kdc[482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 23:14:52 dist-test-slave-754n krb5kdc[482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
W20260923 23:14:52.110391 596 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 23:14:52.104968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58858 (local address 127.24.227.254:34607)
0923 23:14:52.105029 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:52.105031 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:52.105093 (+ 62us) server_negotiation.cc:407] Connection header received
0923 23:14:52.105158 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:52.105158 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:52.105168 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:52.105223 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:52.105692 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:52.106421 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:52.107308 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:52.107608 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:52.107672 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:52.110337 (+ 2665us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58858: BlockingRecv error: recv got EOF from 127.0.0.1:58858 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Sep 23 23:14:52 dist-test-slave-754n krb5kdc[482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205292, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM
I20260923 23:14:52.118547 596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:52.111153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58874 (local address 127.24.227.254:34607)
0923 23:14:52.111186 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:52.111188 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:52.111232 (+ 44us) server_negotiation.cc:407] Connection header received
0923 23:14:52.111298 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:52.111298 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:52.111305 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:52.111346 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:52.111961 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:52.112405 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:52.113332 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:52.113483 (+ 151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:52.113526 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:52.115838 (+ 2312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:52.115839 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:52.115840 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:52.115847 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:52.117432 (+ 1585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:52.117834 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:52.117835 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:52.117835 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:52.117907 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:52.118090 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:52.118091 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:52.118091 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:52.118192 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:52.118288 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:52.118389 (+ 101us) server_negotiation.cc:299] Negotiation successful
0923 23:14:52.118457 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260923 23:14:52.119930 526 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58874:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 23:14:52.120093 526 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 23:14:52.121464 804 tablet_service.cc:1467] Processing CreateTablet for tablet c23f8d729c6040c785def6ed8d4d91fc (DEFAULT_TABLE table=test-table [id=12d62ea76bb14b5f85e1ecc383ffc6e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 23:14:52.121511 944 tablet_service.cc:1467] Processing CreateTablet for tablet c23f8d729c6040c785def6ed8d4d91fc (DEFAULT_TABLE table=test-table [id=12d62ea76bb14b5f85e1ecc383ffc6e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 23:14:52.121503 664 tablet_service.cc:1467] Processing CreateTablet for tablet c23f8d729c6040c785def6ed8d4d91fc (DEFAULT_TABLE table=test-table [id=12d62ea76bb14b5f85e1ecc383ffc6e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 23:14:52.121595 804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c23f8d729c6040c785def6ed8d4d91fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:14:52.121606 944 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c23f8d729c6040c785def6ed8d4d91fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:14:52.121620 664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c23f8d729c6040c785def6ed8d4d91fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:14:52.122901 1035 tablet_bootstrap.cc:492] T c23f8d729c6040c785def6ed8d4d91fc P e9d6ab43fc0f491daef44c6e09be0e4a: Bootstrap starting.
I20260923 23:14:52.122978 1039 tablet_bootstrap.cc:492] T c23f8d729c6040c785def6ed8d4d91fc P 9d239c22857640e7b2ff0c7008fbfaca: Bootstrap starting.
I20260923 23:14:52.123325 1039 tablet_bootstrap.cc:654] T c23f8d729c6040c785def6ed8d4d91fc P 9d239c22857640e7b2ff0c7008fbfaca: Neither blocks nor log segments found. Creating new log.
I20260923 23:14:52.123477 1035 tablet_bootstrap.cc:654] T c23f8d729c6040c785def6ed8d4d91fc P e9d6ab43fc0f491daef44c6e09be0e4a: Neither blocks nor log segments found. Creating new log.
I20260923 23:14:52.123567 1036 tablet_bootstrap.cc:492] T c23f8d729c6040c785def6ed8d4d91fc P 2cd0dd070b8643bf966ede03f18a5de4: Bootstrap starting.
I20260923 23:14:52.123991 1036 tablet_bootstrap.cc:654] T c23f8d729c6040c785def6ed8d4d91fc P 2cd0dd070b8643bf966ede03f18a5de4: Neither blocks nor log segments found. Creating new log.
I20260923 23:14:52.124297 1035 tablet_bootstrap.cc:492] T c23f8d729c6040c785def6ed8d4d91fc P e9d6ab43fc0f491daef44c6e09be0e4a: No bootstrap required, opened a new log
I20260923 23:14:52.124349 1035 ts_tablet_manager.cc:1397] T c23f8d729c6040c785def6ed8d4d91fc P e9d6ab43fc0f491daef44c6e09be0e4a: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260923 23:14:52.124503 1035 raft_consensus.cc:348] T c23f8d729c6040c785def6ed8d4d91fc P e9d6ab43fc0f491daef44c6e09be0e4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cd0dd070b8643bf966ede03f18a5de4" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 43785 } } peers { permanent_uuid: "9d239c22857640e7b2ff0c7008fbfaca" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 38277 } } peers { permanent_uuid: "e9d6ab43fc0f491daef44c6e09be0e4a" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39441 } }
I20260923 23:14:52.124581 1035 raft_consensus.cc:374] T c23f8d729c6040c785def6ed8d4d91fc P e9d6ab43fc0f491daef44c6e09be0e4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:14:52.124588 1036 tablet_bootstrap.cc:492] T c23f8d729c6040c785def6ed8d4d91fc P 2cd0dd070b8643bf966ede03f18a5de4: No bootstrap required, opened a new log
I20260923 23:14:52.124606 1035 raft_consensus.cc:729] T c23f8d729c6040c785def6ed8d4d91fc P e9d6ab43fc0f491daef44c6e09be0e4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9d6ab43fc0f491daef44c6e09be0e4a, State: Initialized, Role: FOLLOWER
I20260923 23:14:52.124629 1036 ts_tablet_manager.cc:1397] T c23f8d729c6040c785def6ed8d4d91fc P 2cd0dd070b8643bf966ede03f18a5de4: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260923 23:14:52.124650 1035 consensus_queue.cc:260] T c23f8d729c6040c785def6ed8d4d91fc P e9d6ab43fc0f491daef44c6e09be0e4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cd0dd070b8643bf966ede03f18a5de4" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 43785 } } peers { permanent_uuid: "9d239c22857640e7b2ff0c7008fbfaca" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 38277 } } peers { permanent_uuid: "e9d6ab43fc0f491daef44c6e09be0e4a" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39441 } }
I20260923 23:14:52.124763 1035 ts_tablet_manager.cc:1428] T c23f8d729c6040c785def6ed8d4d91fc P e9d6ab43fc0f491daef44c6e09be0e4a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:52.124764 1036 raft_consensus.cc:348] T c23f8d729c6040c785def6ed8d4d91fc P 2cd0dd070b8643bf966ede03f18a5de4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cd0dd070b8643bf966ede03f18a5de4" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 43785 } } peers { permanent_uuid: "9d239c22857640e7b2ff0c7008fbfaca" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 38277 } } peers { permanent_uuid: "e9d6ab43fc0f491daef44c6e09be0e4a" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39441 } }
I20260923 23:14:52.124827 1036 raft_consensus.cc:374] T c23f8d729c6040c785def6ed8d4d91fc P 2cd0dd070b8643bf966ede03f18a5de4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:14:52.124853 1036 raft_consensus.cc:729] T c23f8d729c6040c785def6ed8d4d91fc P 2cd0dd070b8643bf966ede03f18a5de4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cd0dd070b8643bf966ede03f18a5de4, State: Initialized, Role: FOLLOWER
I20260923 23:14:52.124905 1039 tablet_bootstrap.cc:492] T c23f8d729c6040c785def6ed8d4d91fc P 9d239c22857640e7b2ff0c7008fbfaca: No bootstrap required, opened a new log
I20260923 23:14:52.124912 1036 consensus_queue.cc:260] T c23f8d729c6040c785def6ed8d4d91fc P 2cd0dd070b8643bf966ede03f18a5de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cd0dd070b8643bf966ede03f18a5de4" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 43785 } } peers { permanent_uuid: "9d239c22857640e7b2ff0c7008fbfaca" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 38277 } } peers { permanent_uuid: "e9d6ab43fc0f491daef44c6e09be0e4a" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39441 } }
I20260923 23:14:52.124961 1039 ts_tablet_manager.cc:1397] T c23f8d729c6040c785def6ed8d4d91fc P 9d239c22857640e7b2ff0c7008fbfaca: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260923 23:14:52.124993 1036 ts_tablet_manager.cc:1428] T c23f8d729c6040c785def6ed8d4d91fc P 2cd0dd070b8643bf966ede03f18a5de4: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260923 23:14:52.125052 1039 raft_consensus.cc:348] T c23f8d729c6040c785def6ed8d4d91fc P 9d239c22857640e7b2ff0c7008fbfaca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cd0dd070b8643bf966ede03f18a5de4" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 43785 } } peers { permanent_uuid: "9d239c22857640e7b2ff0c7008fbfaca" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 38277 } } peers { permanent_uuid: "e9d6ab43fc0f491daef44c6e09be0e4a" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39441 } }
I20260923 23:14:52.125109 1039 raft_consensus.cc:374] T c23f8d729c6040c785def6ed8d4d91fc P 9d239c22857640e7b2ff0c7008fbfaca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:14:52.125128 1039 raft_consensus.cc:729] T c23f8d729c6040c785def6ed8d4d91fc P 9d239c22857640e7b2ff0c7008fbfaca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d239c22857640e7b2ff0c7008fbfaca, State: Initialized, Role: FOLLOWER
I20260923 23:14:52.125164 1039 consensus_queue.cc:260] T c23f8d729c6040c785def6ed8d4d91fc P 9d239c22857640e7b2ff0c7008fbfaca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cd0dd070b8643bf966ede03f18a5de4" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 43785 } } peers { permanent_uuid: "9d239c22857640e7b2ff0c7008fbfaca" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 38277 } } peers { permanent_uuid: "e9d6ab43fc0f491daef44c6e09be0e4a" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39441 } }
I20260923 23:14:52.125236 1039 ts_tablet_manager.cc:1428] T c23f8d729c6040c785def6ed8d4d91fc P 9d239c22857640e7b2ff0c7008fbfaca: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:52.194067 1041 raft_consensus.cc:482] T c23f8d729c6040c785def6ed8d4d91fc P e9d6ab43fc0f491daef44c6e09be0e4a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 23:14:52.194154 1041 raft_consensus.cc:504] T c23f8d729c6040c785def6ed8d4d91fc P e9d6ab43fc0f491daef44c6e09be0e4a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cd0dd070b8643bf966ede03f18a5de4" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 43785 } } peers { permanent_uuid: "9d239c22857640e7b2ff0c7008fbfaca" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 38277 } } peers { permanent_uuid: "e9d6ab43fc0f491daef44c6e09be0e4a" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39441 } }
I20260923 23:14:52.194499 1041 leader_election.cc:288] T c23f8d729c6040c785def6ed8d4d91fc P e9d6ab43fc0f491daef44c6e09be0e4a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2cd0dd070b8643bf966ede03f18a5de4 (127.24.227.193:43785), 9d239c22857640e7b2ff0c7008fbfaca (127.24.227.194:38277)
I20260923 23:14:52.197407 1033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:52.194821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.195:54245 (local address 127.24.227.193:43785)
0923 23:14:52.194884 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:52.194886 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:52.194912 (+ 26us) server_negotiation.cc:407] Connection header received
0923 23:14:52.194971 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:52.194972 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:52.194981 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:52.195019 (+ 38us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:14:52.195161 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:52.195712 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:52.196691 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:52.197051 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:52.197132 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:52.197141 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:52.197304 (+ 163us) server_negotiation.cc:299] Negotiation successful
0923 23:14:52.197341 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260923 23:14:52.197408 903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:52.194758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.193:43785 (local address 127.24.227.195:54245)
0923 23:14:52.194835 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:52.194863 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:52.194910 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:52.195029 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:52.195029 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:52.195031 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:14:52.195097 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:52.195099 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:52.195799 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:52.195800 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:52.196599 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:52.196601 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:52.197141 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:52.197142 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:52.197226 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:52.197229 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 23:14:52.197299 (+ 70us) client_negotiation.cc:240] Negotiation successful
0923 23:14:52.197339 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59,"mutex_wait_us":15}
I20260923 23:14:52.197635 684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c23f8d729c6040c785def6ed8d4d91fc" candidate_uuid: "e9d6ab43fc0f491daef44c6e09be0e4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cd0dd070b8643bf966ede03f18a5de4" is_pre_election: true
I20260923 23:14:52.197717 684 raft_consensus.cc:2454] T c23f8d729c6040c785def6ed8d4d91fc P 2cd0dd070b8643bf966ede03f18a5de4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9d6ab43fc0f491daef44c6e09be0e4a in term 0.
I20260923 23:14:52.197774 1031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:52.194940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.195:40613 (local address 127.24.227.194:38277)
0923 23:14:52.194991 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:52.194993 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:52.195041 (+ 48us) server_negotiation.cc:407] Connection header received
0923 23:14:52.195105 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:52.195106 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:52.195137 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:52.195187 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:14:52.195406 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:52.195833 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:52.196932 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:52.197293 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:52.197380 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:52.197390 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:52.197678 (+ 288us) server_negotiation.cc:299] Negotiation successful
0923 23:14:52.197718 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260923 23:14:52.197780 1106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:52.194881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.194:38277 (local address 127.24.227.195:40613)
0923 23:14:52.194995 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:52.195004 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:52.195055 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:52.195221 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:52.195222 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:52.195223 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:14:52.195316 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:52.195319 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:52.195954 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:52.195955 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:52.196839 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:52.196842 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:52.197393 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:52.197395 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:52.197595 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:52.197601 (+ 6us) client_negotiation.cc:769] Sending connection context
0923 23:14:52.197668 (+ 67us) client_negotiation.cc:240] Negotiation successful
0923 23:14:52.197703 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":65,"threads_started":1}
I20260923 23:14:52.197870 897 leader_election.cc:302] T c23f8d729c6040c785def6ed8d4d91fc P e9d6ab43fc0f491daef44c6e09be0e4a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cd0dd070b8643bf966ede03f18a5de4, e9d6ab43fc0f491daef44c6e09be0e4a; no voters:
I20260923 23:14:52.198007 1041 raft_consensus.cc:2781] T c23f8d729c6040c785def6ed8d4d91fc P e9d6ab43fc0f491daef44c6e09be0e4a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 23:14:52.198058 1041 raft_consensus.cc:482] T c23f8d729c6040c785def6ed8d4d91fc P e9d6ab43fc0f491daef44c6e09be0e4a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 23:14:52.198087 1041 raft_consensus.cc:3037] T c23f8d729c6040c785def6ed8d4d91fc P e9d6ab43fc0f491daef44c6e09be0e4a [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:14:52.198163 824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c23f8d729c6040c785def6ed8d4d91fc" candidate_uuid: "e9d6ab43fc0f491daef44c6e09be0e4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d239c22857640e7b2ff0c7008fbfaca" is_pre_election: true
I20260923 23:14:52.198254 824 raft_consensus.cc:2454] T c23f8d729c6040c785def6ed8d4d91fc P 9d239c22857640e7b2ff0c7008fbfaca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9d6ab43fc0f491daef44c6e09be0e4a in term 0.
I20260923 23:14:52.198818 1041 raft_consensus.cc:504] T c23f8d729c6040c785def6ed8d4d91fc P e9d6ab43fc0f491daef44c6e09be0e4a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cd0dd070b8643bf966ede03f18a5de4" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 43785 } } peers { permanent_uuid: "9d239c22857640e7b2ff0c7008fbfaca" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 38277 } } peers { permanent_uuid: "e9d6ab43fc0f491daef44c6e09be0e4a" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39441 } }
I20260923 23:14:52.198942 1041 leader_election.cc:288] T c23f8d729c6040c785def6ed8d4d91fc P e9d6ab43fc0f491daef44c6e09be0e4a [CANDIDATE]: Term 1 election: Requested vote from peers 2cd0dd070b8643bf966ede03f18a5de4 (127.24.227.193:43785), 9d239c22857640e7b2ff0c7008fbfaca (127.24.227.194:38277)
I20260923 23:14:52.199091 684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c23f8d729c6040c785def6ed8d4d91fc" candidate_uuid: "e9d6ab43fc0f491daef44c6e09be0e4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cd0dd070b8643bf966ede03f18a5de4"
I20260923 23:14:52.199117 824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c23f8d729c6040c785def6ed8d4d91fc" candidate_uuid: "e9d6ab43fc0f491daef44c6e09be0e4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d239c22857640e7b2ff0c7008fbfaca"
I20260923 23:14:52.199175 684 raft_consensus.cc:3037] T c23f8d729c6040c785def6ed8d4d91fc P 2cd0dd070b8643bf966ede03f18a5de4 [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:14:52.199177 824 raft_consensus.cc:3037] T c23f8d729c6040c785def6ed8d4d91fc P 9d239c22857640e7b2ff0c7008fbfaca [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:14:52.199709 684 raft_consensus.cc:2454] T c23f8d729c6040c785def6ed8d4d91fc P 2cd0dd070b8643bf966ede03f18a5de4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9d6ab43fc0f491daef44c6e09be0e4a in term 1.
I20260923 23:14:52.199759 824 raft_consensus.cc:2454] T c23f8d729c6040c785def6ed8d4d91fc P 9d239c22857640e7b2ff0c7008fbfaca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9d6ab43fc0f491daef44c6e09be0e4a in term 1.
I20260923 23:14:52.199913 897 leader_election.cc:302] T c23f8d729c6040c785def6ed8d4d91fc P e9d6ab43fc0f491daef44c6e09be0e4a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d239c22857640e7b2ff0c7008fbfaca, e9d6ab43fc0f491daef44c6e09be0e4a; no voters:
I20260923 23:14:52.200019 1041 raft_consensus.cc:2781] T c23f8d729c6040c785def6ed8d4d91fc P e9d6ab43fc0f491daef44c6e09be0e4a [term 1 FOLLOWER]: Leader election won for term 1
I20260923 23:14:52.200160 1041 raft_consensus.cc:686] T c23f8d729c6040c785def6ed8d4d91fc P e9d6ab43fc0f491daef44c6e09be0e4a [term 1 LEADER]: Becoming Leader. State: Replica: e9d6ab43fc0f491daef44c6e09be0e4a, State: Running, Role: LEADER
I20260923 23:14:52.200237 1041 consensus_queue.cc:237] T c23f8d729c6040c785def6ed8d4d91fc P e9d6ab43fc0f491daef44c6e09be0e4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cd0dd070b8643bf966ede03f18a5de4" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 43785 } } peers { permanent_uuid: "9d239c22857640e7b2ff0c7008fbfaca" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 38277 } } peers { permanent_uuid: "e9d6ab43fc0f491daef44c6e09be0e4a" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39441 } }
I20260923 23:14:52.201007 527 catalog_manager.cc:5495] T c23f8d729c6040c785def6ed8d4d91fc P e9d6ab43fc0f491daef44c6e09be0e4a reported cstate change: term changed from 0 to 1, leader changed from <none> to e9d6ab43fc0f491daef44c6e09be0e4a (127.24.227.195). New cstate: current_term: 1 leader_uuid: "e9d6ab43fc0f491daef44c6e09be0e4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cd0dd070b8643bf966ede03f18a5de4" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 43785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d239c22857640e7b2ff0c7008fbfaca" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 38277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9d6ab43fc0f491daef44c6e09be0e4a" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39441 } health_report { overall_health: HEALTHY } } }
I20260923 23:14:52.217366 1034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:52.215233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43310 (local address 127.24.227.195:39441)
0923 23:14:52.215294 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:52.215295 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:52.215302 (+ 7us) server_negotiation.cc:407] Connection header received
0923 23:14:52.215352 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:52.215353 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:52.215367 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:52.215424 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 23:14:52.215522 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:52.216003 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:52.216703 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:52.216882 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:52.216933 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:52.217117 (+ 184us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 23:14:52.217208 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 23:14:52.217269 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:52.217307 (+ 38us) server_negotiation.cc:299] Negotiation successful
0923 23:14:52.217320 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260923 23:14:52.218264 684 raft_consensus.cc:1261] T c23f8d729c6040c785def6ed8d4d91fc P 2cd0dd070b8643bf966ede03f18a5de4 [term 1 FOLLOWER]: Refusing update from remote peer e9d6ab43fc0f491daef44c6e09be0e4a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 23:14:52.218302 824 raft_consensus.cc:1261] T c23f8d729c6040c785def6ed8d4d91fc P 9d239c22857640e7b2ff0c7008fbfaca [term 1 FOLLOWER]: Refusing update from remote peer e9d6ab43fc0f491daef44c6e09be0e4a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 23:14:52.218555 1041 consensus_queue.cc:1035] T c23f8d729c6040c785def6ed8d4d91fc P e9d6ab43fc0f491daef44c6e09be0e4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d239c22857640e7b2ff0c7008fbfaca" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 38277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 23:14:52.218688 1107 consensus_queue.cc:1035] T c23f8d729c6040c785def6ed8d4d91fc P e9d6ab43fc0f491daef44c6e09be0e4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2cd0dd070b8643bf966ede03f18a5de4" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 43785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 23:14:52.221505 527 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58874:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 23:14:52.221588 527 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58874:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 23:14:52.222122 527 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 607f68c4a2fb4f858b83bbc07edd2bdb: Sending DeleteTablet for 3 replicas of tablet c23f8d729c6040c785def6ed8d4d91fc
I20260923 23:14:52.222366 944 tablet_service.cc:1514] Processing DeleteTablet for tablet c23f8d729c6040c785def6ed8d4d91fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 23:14:52 UTC) from {username='oryx', principal='oryx/127.24.227.254@KRBTEST.COM'} at 127.0.0.1:43294
I20260923 23:14:52.222424 664 tablet_service.cc:1514] Processing DeleteTablet for tablet c23f8d729c6040c785def6ed8d4d91fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 23:14:52 UTC) from {username='oryx', principal='oryx/127.24.227.254@KRBTEST.COM'} at 127.0.0.1:57858
I20260923 23:14:52.222450 1057 tablet_replica.cc:330] T c23f8d729c6040c785def6ed8d4d91fc P e9d6ab43fc0f491daef44c6e09be0e4a: stopping tablet replica
I20260923 23:14:52.222467 804 tablet_service.cc:1514] Processing DeleteTablet for tablet c23f8d729c6040c785def6ed8d4d91fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 23:14:52 UTC) from {username='oryx', principal='oryx/127.24.227.254@KRBTEST.COM'} at 127.0.0.1:35736
I20260923 23:14:52.222491 1057 raft_consensus.cc:2229] T c23f8d729c6040c785def6ed8d4d91fc P e9d6ab43fc0f491daef44c6e09be0e4a [term 1 LEADER]: Raft consensus shutting down.
I20260923 23:14:52.222499 1058 tablet_replica.cc:330] T c23f8d729c6040c785def6ed8d4d91fc P 2cd0dd070b8643bf966ede03f18a5de4: stopping tablet replica
I20260923 23:14:52.222534 1059 tablet_replica.cc:330] T c23f8d729c6040c785def6ed8d4d91fc P 9d239c22857640e7b2ff0c7008fbfaca: stopping tablet replica
I20260923 23:14:52.222541 1058 raft_consensus.cc:2229] T c23f8d729c6040c785def6ed8d4d91fc P 2cd0dd070b8643bf966ede03f18a5de4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 23:14:52.222551 1057 raft_consensus.cc:2258] T c23f8d729c6040c785def6ed8d4d91fc P e9d6ab43fc0f491daef44c6e09be0e4a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 23:14:52.222572 1059 raft_consensus.cc:2229] T c23f8d729c6040c785def6ed8d4d91fc P 9d239c22857640e7b2ff0c7008fbfaca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 23:14:52.222580 1058 raft_consensus.cc:2258] T c23f8d729c6040c785def6ed8d4d91fc P 2cd0dd070b8643bf966ede03f18a5de4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 23:14:52.222610 1059 raft_consensus.cc:2258] T c23f8d729c6040c785def6ed8d4d91fc P 9d239c22857640e7b2ff0c7008fbfaca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 23:14:52.222882 1058 ts_tablet_manager.cc:1905] T c23f8d729c6040c785def6ed8d4d91fc P 2cd0dd070b8643bf966ede03f18a5de4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 23:14:52.222945 527 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58874:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 23:14:52.223083 527 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 23:14:52.223492 1059 ts_tablet_manager.cc:1905] T c23f8d729c6040c785def6ed8d4d91fc P 9d239c22857640e7b2ff0c7008fbfaca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 23:14:52.223634 1057 ts_tablet_manager.cc:1905] T c23f8d729c6040c785def6ed8d4d91fc P e9d6ab43fc0f491daef44c6e09be0e4a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 23:14:52.224426 1058 ts_tablet_manager.cc:1918] T c23f8d729c6040c785def6ed8d4d91fc P 2cd0dd070b8643bf966ede03f18a5de4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 23:14:52.224488 1058 log.cc:1196] T c23f8d729c6040c785def6ed8d4d91fc P 2cd0dd070b8643bf966ede03f18a5de4: Deleting WAL directory at /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/minicluster-data/ts-0/wal/wals/c23f8d729c6040c785def6ed8d4d91fc
I20260923 23:14:52.224702 1058 ts_tablet_manager.cc:1939] T c23f8d729c6040c785def6ed8d4d91fc P 2cd0dd070b8643bf966ede03f18a5de4: Deleting consensus metadata
I20260923 23:14:52.225136 1059 ts_tablet_manager.cc:1918] T c23f8d729c6040c785def6ed8d4d91fc P 9d239c22857640e7b2ff0c7008fbfaca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 23:14:52.225183 1059 log.cc:1196] T c23f8d729c6040c785def6ed8d4d91fc P 9d239c22857640e7b2ff0c7008fbfaca: Deleting WAL directory at /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/minicluster-data/ts-1/wal/wals/c23f8d729c6040c785def6ed8d4d91fc
I20260923 23:14:52.225277 513 catalog_manager.cc:4841] TS 2cd0dd070b8643bf966ede03f18a5de4 (127.24.227.193:43785): tablet c23f8d729c6040c785def6ed8d4d91fc (table test-table [id=12d62ea76bb14b5f85e1ecc383ffc6e7]) successfully deleted
I20260923 23:14:52.225308 1057 ts_tablet_manager.cc:1918] T c23f8d729c6040c785def6ed8d4d91fc P e9d6ab43fc0f491daef44c6e09be0e4a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 23:14:52.225355 1057 log.cc:1196] T c23f8d729c6040c785def6ed8d4d91fc P e9d6ab43fc0f491daef44c6e09be0e4a: Deleting WAL directory at /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/minicluster-data/ts-2/wal/wals/c23f8d729c6040c785def6ed8d4d91fc
I20260923 23:14:52.225411 1059 ts_tablet_manager.cc:1939] T c23f8d729c6040c785def6ed8d4d91fc P 9d239c22857640e7b2ff0c7008fbfaca: Deleting consensus metadata
I20260923 23:14:52.225595 1057 ts_tablet_manager.cc:1939] T c23f8d729c6040c785def6ed8d4d91fc P e9d6ab43fc0f491daef44c6e09be0e4a: Deleting consensus metadata
I20260923 23:14:52.225919 511 catalog_manager.cc:4841] TS e9d6ab43fc0f491daef44c6e09be0e4a (127.24.227.195:39441): tablet c23f8d729c6040c785def6ed8d4d91fc (table test-table [id=12d62ea76bb14b5f85e1ecc383ffc6e7]) successfully deleted
I20260923 23:14:52.226353 513 catalog_manager.cc:4841] TS 9d239c22857640e7b2ff0c7008fbfaca (127.24.227.194:38277): tablet c23f8d729c6040c785def6ed8d4d91fc (table test-table [id=12d62ea76bb14b5f85e1ecc383ffc6e7]) successfully deleted
I20260923 23:14:52.226776 944 tablet_service.cc:1467] Processing CreateTablet for tablet 71ad1531269a4c1e97b8907597b4046c (DEFAULT_TABLE table=test-table [id=da45c7c5e7354fbc888b7aefe516c98a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 23:14:52.226774 804 tablet_service.cc:1467] Processing CreateTablet for tablet 71ad1531269a4c1e97b8907597b4046c (DEFAULT_TABLE table=test-table [id=da45c7c5e7354fbc888b7aefe516c98a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 23:14:52.226876 944 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 71ad1531269a4c1e97b8907597b4046c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:14:52.226877 804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 71ad1531269a4c1e97b8907597b4046c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:14:52.227936 1039 tablet_bootstrap.cc:492] T 71ad1531269a4c1e97b8907597b4046c P 9d239c22857640e7b2ff0c7008fbfaca: Bootstrap starting.
I20260923 23:14:52.227985 664 tablet_service.cc:1467] Processing CreateTablet for tablet 71ad1531269a4c1e97b8907597b4046c (DEFAULT_TABLE table=test-table [id=da45c7c5e7354fbc888b7aefe516c98a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 23:14:52.228062 664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 71ad1531269a4c1e97b8907597b4046c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:14:52.228222 1039 tablet_bootstrap.cc:654] T 71ad1531269a4c1e97b8907597b4046c P 9d239c22857640e7b2ff0c7008fbfaca: Neither blocks nor log segments found. Creating new log.
I20260923 23:14:52.229106 1036 tablet_bootstrap.cc:492] T 71ad1531269a4c1e97b8907597b4046c P 2cd0dd070b8643bf966ede03f18a5de4: Bootstrap starting.
I20260923 23:14:52.229187 1035 tablet_bootstrap.cc:492] T 71ad1531269a4c1e97b8907597b4046c P e9d6ab43fc0f491daef44c6e09be0e4a: Bootstrap starting.
I20260923 23:14:52.229396 1036 tablet_bootstrap.cc:654] T 71ad1531269a4c1e97b8907597b4046c P 2cd0dd070b8643bf966ede03f18a5de4: Neither blocks nor log segments found. Creating new log.
I20260923 23:14:52.229625 1035 tablet_bootstrap.cc:654] T 71ad1531269a4c1e97b8907597b4046c P e9d6ab43fc0f491daef44c6e09be0e4a: Neither blocks nor log segments found. Creating new log.
I20260923 23:14:52.229961 1036 tablet_bootstrap.cc:492] T 71ad1531269a4c1e97b8907597b4046c P 2cd0dd070b8643bf966ede03f18a5de4: No bootstrap required, opened a new log
I20260923 23:14:52.230006 1036 ts_tablet_manager.cc:1397] T 71ad1531269a4c1e97b8907597b4046c P 2cd0dd070b8643bf966ede03f18a5de4: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260923 23:14:52.230118 1036 raft_consensus.cc:348] T 71ad1531269a4c1e97b8907597b4046c P 2cd0dd070b8643bf966ede03f18a5de4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cd0dd070b8643bf966ede03f18a5de4" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 43785 } } peers { permanent_uuid: "9d239c22857640e7b2ff0c7008fbfaca" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 38277 } } peers { permanent_uuid: "e9d6ab43fc0f491daef44c6e09be0e4a" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39441 } }
I20260923 23:14:52.230156 1039 tablet_bootstrap.cc:492] T 71ad1531269a4c1e97b8907597b4046c P 9d239c22857640e7b2ff0c7008fbfaca: No bootstrap required, opened a new log
I20260923 23:14:52.230180 1036 raft_consensus.cc:374] T 71ad1531269a4c1e97b8907597b4046c P 2cd0dd070b8643bf966ede03f18a5de4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:14:52.230198 1036 raft_consensus.cc:729] T 71ad1531269a4c1e97b8907597b4046c P 2cd0dd070b8643bf966ede03f18a5de4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cd0dd070b8643bf966ede03f18a5de4, State: Initialized, Role: FOLLOWER
I20260923 23:14:52.230198 1039 ts_tablet_manager.cc:1397] T 71ad1531269a4c1e97b8907597b4046c P 9d239c22857640e7b2ff0c7008fbfaca: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260923 23:14:52.230232 1036 consensus_queue.cc:260] T 71ad1531269a4c1e97b8907597b4046c P 2cd0dd070b8643bf966ede03f18a5de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cd0dd070b8643bf966ede03f18a5de4" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 43785 } } peers { permanent_uuid: "9d239c22857640e7b2ff0c7008fbfaca" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 38277 } } peers { permanent_uuid: "e9d6ab43fc0f491daef44c6e09be0e4a" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39441 } }
I20260923 23:14:52.230321 1036 ts_tablet_manager.cc:1428] T 71ad1531269a4c1e97b8907597b4046c P 2cd0dd070b8643bf966ede03f18a5de4: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:52.230293 1039 raft_consensus.cc:348] T 71ad1531269a4c1e97b8907597b4046c P 9d239c22857640e7b2ff0c7008fbfaca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cd0dd070b8643bf966ede03f18a5de4" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 43785 } } peers { permanent_uuid: "9d239c22857640e7b2ff0c7008fbfaca" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 38277 } } peers { permanent_uuid: "e9d6ab43fc0f491daef44c6e09be0e4a" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39441 } }
I20260923 23:14:52.230355 1039 raft_consensus.cc:374] T 71ad1531269a4c1e97b8907597b4046c P 9d239c22857640e7b2ff0c7008fbfaca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:14:52.230381 1039 raft_consensus.cc:729] T 71ad1531269a4c1e97b8907597b4046c P 9d239c22857640e7b2ff0c7008fbfaca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d239c22857640e7b2ff0c7008fbfaca, State: Initialized, Role: FOLLOWER
I20260923 23:14:52.230424 1039 consensus_queue.cc:260] T 71ad1531269a4c1e97b8907597b4046c P 9d239c22857640e7b2ff0c7008fbfaca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cd0dd070b8643bf966ede03f18a5de4" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 43785 } } peers { permanent_uuid: "9d239c22857640e7b2ff0c7008fbfaca" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 38277 } } peers { permanent_uuid: "e9d6ab43fc0f491daef44c6e09be0e4a" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39441 } }
I20260923 23:14:52.230505 1039 ts_tablet_manager.cc:1428] T 71ad1531269a4c1e97b8907597b4046c P 9d239c22857640e7b2ff0c7008fbfaca: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:52.230691 1035 tablet_bootstrap.cc:492] T 71ad1531269a4c1e97b8907597b4046c P e9d6ab43fc0f491daef44c6e09be0e4a: No bootstrap required, opened a new log
I20260923 23:14:52.230728 1035 ts_tablet_manager.cc:1397] T 71ad1531269a4c1e97b8907597b4046c P e9d6ab43fc0f491daef44c6e09be0e4a: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260923 23:14:52.230855 1035 raft_consensus.cc:348] T 71ad1531269a4c1e97b8907597b4046c P e9d6ab43fc0f491daef44c6e09be0e4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cd0dd070b8643bf966ede03f18a5de4" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 43785 } } peers { permanent_uuid: "9d239c22857640e7b2ff0c7008fbfaca" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 38277 } } peers { permanent_uuid: "e9d6ab43fc0f491daef44c6e09be0e4a" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39441 } }
I20260923 23:14:52.230931 1035 raft_consensus.cc:374] T 71ad1531269a4c1e97b8907597b4046c P e9d6ab43fc0f491daef44c6e09be0e4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:14:52.230955 1035 raft_consensus.cc:729] T 71ad1531269a4c1e97b8907597b4046c P e9d6ab43fc0f491daef44c6e09be0e4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9d6ab43fc0f491daef44c6e09be0e4a, State: Initialized, Role: FOLLOWER
I20260923 23:14:52.230991 1035 consensus_queue.cc:260] T 71ad1531269a4c1e97b8907597b4046c P e9d6ab43fc0f491daef44c6e09be0e4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cd0dd070b8643bf966ede03f18a5de4" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 43785 } } peers { permanent_uuid: "9d239c22857640e7b2ff0c7008fbfaca" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 38277 } } peers { permanent_uuid: "e9d6ab43fc0f491daef44c6e09be0e4a" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39441 } }
I20260923 23:14:52.231069 1035 ts_tablet_manager.cc:1428] T 71ad1531269a4c1e97b8907597b4046c P e9d6ab43fc0f491daef44c6e09be0e4a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:52.310736 1089 raft_consensus.cc:482] T 71ad1531269a4c1e97b8907597b4046c P 9d239c22857640e7b2ff0c7008fbfaca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 23:14:52.310863 1089 raft_consensus.cc:504] T 71ad1531269a4c1e97b8907597b4046c P 9d239c22857640e7b2ff0c7008fbfaca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cd0dd070b8643bf966ede03f18a5de4" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 43785 } } peers { permanent_uuid: "9d239c22857640e7b2ff0c7008fbfaca" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 38277 } } peers { permanent_uuid: "e9d6ab43fc0f491daef44c6e09be0e4a" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39441 } }
I20260923 23:14:52.311064 1089 leader_election.cc:288] T 71ad1531269a4c1e97b8907597b4046c P 9d239c22857640e7b2ff0c7008fbfaca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2cd0dd070b8643bf966ede03f18a5de4 (127.24.227.193:43785), e9d6ab43fc0f491daef44c6e09be0e4a (127.24.227.195:39441)
I20260923 23:14:52.311244 963 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71ad1531269a4c1e97b8907597b4046c" candidate_uuid: "9d239c22857640e7b2ff0c7008fbfaca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9d6ab43fc0f491daef44c6e09be0e4a" is_pre_election: true
I20260923 23:14:52.311286 684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71ad1531269a4c1e97b8907597b4046c" candidate_uuid: "9d239c22857640e7b2ff0c7008fbfaca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cd0dd070b8643bf966ede03f18a5de4" is_pre_election: true
I20260923 23:14:52.311335 963 raft_consensus.cc:2454] T 71ad1531269a4c1e97b8907597b4046c P e9d6ab43fc0f491daef44c6e09be0e4a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d239c22857640e7b2ff0c7008fbfaca in term 0.
I20260923 23:14:52.311363 684 raft_consensus.cc:2454] T 71ad1531269a4c1e97b8907597b4046c P 2cd0dd070b8643bf966ede03f18a5de4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d239c22857640e7b2ff0c7008fbfaca in term 0.
I20260923 23:14:52.311476 755 leader_election.cc:302] T 71ad1531269a4c1e97b8907597b4046c P 9d239c22857640e7b2ff0c7008fbfaca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d239c22857640e7b2ff0c7008fbfaca, e9d6ab43fc0f491daef44c6e09be0e4a; no voters:
I20260923 23:14:52.311575 1089 raft_consensus.cc:2781] T 71ad1531269a4c1e97b8907597b4046c P 9d239c22857640e7b2ff0c7008fbfaca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 23:14:52.311610 1089 raft_consensus.cc:482] T 71ad1531269a4c1e97b8907597b4046c P 9d239c22857640e7b2ff0c7008fbfaca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 23:14:52.311661 1089 raft_consensus.cc:3037] T 71ad1531269a4c1e97b8907597b4046c P 9d239c22857640e7b2ff0c7008fbfaca [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:14:52.312423 1089 raft_consensus.cc:504] T 71ad1531269a4c1e97b8907597b4046c P 9d239c22857640e7b2ff0c7008fbfaca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cd0dd070b8643bf966ede03f18a5de4" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 43785 } } peers { permanent_uuid: "9d239c22857640e7b2ff0c7008fbfaca" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 38277 } } peers { permanent_uuid: "e9d6ab43fc0f491daef44c6e09be0e4a" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39441 } }
I20260923 23:14:52.312552 1089 leader_election.cc:288] T 71ad1531269a4c1e97b8907597b4046c P 9d239c22857640e7b2ff0c7008fbfaca [CANDIDATE]: Term 1 election: Requested vote from peers 2cd0dd070b8643bf966ede03f18a5de4 (127.24.227.193:43785), e9d6ab43fc0f491daef44c6e09be0e4a (127.24.227.195:39441)
I20260923 23:14:52.312667 684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71ad1531269a4c1e97b8907597b4046c" candidate_uuid: "9d239c22857640e7b2ff0c7008fbfaca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cd0dd070b8643bf966ede03f18a5de4"
I20260923 23:14:52.312747 684 raft_consensus.cc:3037] T 71ad1531269a4c1e97b8907597b4046c P 2cd0dd070b8643bf966ede03f18a5de4 [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:14:52.312749 963 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71ad1531269a4c1e97b8907597b4046c" candidate_uuid: "9d239c22857640e7b2ff0c7008fbfaca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9d6ab43fc0f491daef44c6e09be0e4a"
I20260923 23:14:52.312839 963 raft_consensus.cc:3037] T 71ad1531269a4c1e97b8907597b4046c P e9d6ab43fc0f491daef44c6e09be0e4a [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:14:52.313306 684 raft_consensus.cc:2454] T 71ad1531269a4c1e97b8907597b4046c P 2cd0dd070b8643bf966ede03f18a5de4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d239c22857640e7b2ff0c7008fbfaca in term 1.
I20260923 23:14:52.313576 963 raft_consensus.cc:2454] T 71ad1531269a4c1e97b8907597b4046c P e9d6ab43fc0f491daef44c6e09be0e4a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d239c22857640e7b2ff0c7008fbfaca in term 1.
I20260923 23:14:52.313748 757 leader_election.cc:302] T 71ad1531269a4c1e97b8907597b4046c P 9d239c22857640e7b2ff0c7008fbfaca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cd0dd070b8643bf966ede03f18a5de4, 9d239c22857640e7b2ff0c7008fbfaca; no voters:
I20260923 23:14:52.313861 1089 raft_consensus.cc:2781] T 71ad1531269a4c1e97b8907597b4046c P 9d239c22857640e7b2ff0c7008fbfaca [term 1 FOLLOWER]: Leader election won for term 1
I20260923 23:14:52.313910 1089 raft_consensus.cc:686] T 71ad1531269a4c1e97b8907597b4046c P 9d239c22857640e7b2ff0c7008fbfaca [term 1 LEADER]: Becoming Leader. State: Replica: 9d239c22857640e7b2ff0c7008fbfaca, State: Running, Role: LEADER
I20260923 23:14:52.313958 1089 consensus_queue.cc:237] T 71ad1531269a4c1e97b8907597b4046c P 9d239c22857640e7b2ff0c7008fbfaca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cd0dd070b8643bf966ede03f18a5de4" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 43785 } } peers { permanent_uuid: "9d239c22857640e7b2ff0c7008fbfaca" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 38277 } } peers { permanent_uuid: "e9d6ab43fc0f491daef44c6e09be0e4a" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39441 } }
I20260923 23:14:52.314594 527 catalog_manager.cc:5495] T 71ad1531269a4c1e97b8907597b4046c P 9d239c22857640e7b2ff0c7008fbfaca reported cstate change: term changed from 0 to 1, leader changed from <none> to 9d239c22857640e7b2ff0c7008fbfaca (127.24.227.194). New cstate: current_term: 1 leader_uuid: "9d239c22857640e7b2ff0c7008fbfaca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cd0dd070b8643bf966ede03f18a5de4" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 43785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d239c22857640e7b2ff0c7008fbfaca" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 38277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9d6ab43fc0f491daef44c6e09be0e4a" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39441 } health_report { overall_health: UNKNOWN } } }
I20260923 23:14:52.323800 1031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:52.321437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35746 (local address 127.24.227.194:38277)
0923 23:14:52.321475 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:52.321477 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:52.321487 (+ 10us) server_negotiation.cc:407] Connection header received
0923 23:14:52.321541 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:52.321542 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:52.321551 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:52.321609 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 23:14:52.321703 (+ 94us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:52.322416 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:52.323081 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:52.323381 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:52.323452 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:52.323553 (+ 101us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 23:14:52.323582 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 23:14:52.323621 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:52.323683 (+ 62us) server_negotiation.cc:299] Negotiation successful
0923 23:14:52.323704 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 23:14:52.325897 963 raft_consensus.cc:1261] T 71ad1531269a4c1e97b8907597b4046c P e9d6ab43fc0f491daef44c6e09be0e4a [term 1 FOLLOWER]: Refusing update from remote peer 9d239c22857640e7b2ff0c7008fbfaca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 23:14:52.325925 684 raft_consensus.cc:1261] T 71ad1531269a4c1e97b8907597b4046c P 2cd0dd070b8643bf966ede03f18a5de4 [term 1 FOLLOWER]: Refusing update from remote peer 9d239c22857640e7b2ff0c7008fbfaca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 23:14:52.326051 1089 consensus_queue.cc:1035] T 71ad1531269a4c1e97b8907597b4046c P 9d239c22857640e7b2ff0c7008fbfaca [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9d6ab43fc0f491daef44c6e09be0e4a" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 39441 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 23:14:52.326128 1045 consensus_queue.cc:1035] T 71ad1531269a4c1e97b8907597b4046c P 9d239c22857640e7b2ff0c7008fbfaca [LEADER]: Connected to new peer: Peer: permanent_uuid: "2cd0dd070b8643bf966ede03f18a5de4" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 43785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 23:14:52.335543 525 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58874:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 23:14:52.335628 525 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58874:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 23:14:52.336282 525 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 607f68c4a2fb4f858b83bbc07edd2bdb: Sending DeleteTablet for 3 replicas of tablet 71ad1531269a4c1e97b8907597b4046c
I20260923 23:14:52.336550 944 tablet_service.cc:1514] Processing DeleteTablet for tablet 71ad1531269a4c1e97b8907597b4046c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 23:14:52 UTC) from {username='oryx', principal='oryx/127.24.227.254@KRBTEST.COM'} at 127.0.0.1:43294
I20260923 23:14:52.336576 664 tablet_service.cc:1514] Processing DeleteTablet for tablet 71ad1531269a4c1e97b8907597b4046c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 23:14:52 UTC) from {username='oryx', principal='oryx/127.24.227.254@KRBTEST.COM'} at 127.0.0.1:57858
I20260923 23:14:52.336593 804 tablet_service.cc:1514] Processing DeleteTablet for tablet 71ad1531269a4c1e97b8907597b4046c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 23:14:52 UTC) from {username='oryx', principal='oryx/127.24.227.254@KRBTEST.COM'} at 127.0.0.1:35736
I20260923 23:14:52.336642 1057 tablet_replica.cc:330] T 71ad1531269a4c1e97b8907597b4046c P e9d6ab43fc0f491daef44c6e09be0e4a: stopping tablet replica
I20260923 23:14:52.336649 1058 tablet_replica.cc:330] T 71ad1531269a4c1e97b8907597b4046c P 2cd0dd070b8643bf966ede03f18a5de4: stopping tablet replica
I20260923 23:14:52.336669 1059 tablet_replica.cc:330] T 71ad1531269a4c1e97b8907597b4046c P 9d239c22857640e7b2ff0c7008fbfaca: stopping tablet replica
I20260923 23:14:52.336688 1057 raft_consensus.cc:2229] T 71ad1531269a4c1e97b8907597b4046c P e9d6ab43fc0f491daef44c6e09be0e4a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 23:14:52.336692 1058 raft_consensus.cc:2229] T 71ad1531269a4c1e97b8907597b4046c P 2cd0dd070b8643bf966ede03f18a5de4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 23:14:52.336706 1059 raft_consensus.cc:2229] T 71ad1531269a4c1e97b8907597b4046c P 9d239c22857640e7b2ff0c7008fbfaca [term 1 LEADER]: Raft consensus shutting down.
I20260923 23:14:52.336723 1057 raft_consensus.cc:2258] T 71ad1531269a4c1e97b8907597b4046c P e9d6ab43fc0f491daef44c6e09be0e4a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 23:14:52.336725 1058 raft_consensus.cc:2258] T 71ad1531269a4c1e97b8907597b4046c P 2cd0dd070b8643bf966ede03f18a5de4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 23:14:52.336750 1059 raft_consensus.cc:2258] T 71ad1531269a4c1e97b8907597b4046c P 9d239c22857640e7b2ff0c7008fbfaca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 23:14:52.337105 1058 ts_tablet_manager.cc:1905] T 71ad1531269a4c1e97b8907597b4046c P 2cd0dd070b8643bf966ede03f18a5de4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 23:14:52.337136 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 599
I20260923 23:14:52.337206 1057 ts_tablet_manager.cc:1905] T 71ad1531269a4c1e97b8907597b4046c P e9d6ab43fc0f491daef44c6e09be0e4a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 23:14:52.338837 1057 ts_tablet_manager.cc:1918] T 71ad1531269a4c1e97b8907597b4046c P e9d6ab43fc0f491daef44c6e09be0e4a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 23:14:52.338907 1057 log.cc:1196] T 71ad1531269a4c1e97b8907597b4046c P e9d6ab43fc0f491daef44c6e09be0e4a: Deleting WAL directory at /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790205259943518-25487-0/minicluster-data/ts-2/wal/wals/71ad1531269a4c1e97b8907597b4046c
I20260923 23:14:52.339143 1057 ts_tablet_manager.cc:1939] T 71ad1531269a4c1e97b8907597b4046c P e9d6ab43fc0f491daef44c6e09be0e4a: Deleting consensus metadata
I20260923 23:14:52.339455 511 catalog_manager.cc:4841] TS e9d6ab43fc0f491daef44c6e09be0e4a (127.24.227.195:39441): tablet 71ad1531269a4c1e97b8907597b4046c (table test-table [id=da45c7c5e7354fbc888b7aefe516c98a]) successfully deleted
I20260923 23:14:52.344189 1059 ts_tablet_manager.cc:1905] T 71ad1531269a4c1e97b8907597b4046c P 9d239c22857640e7b2ff0c7008fbfaca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 23:14:52.344659 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 739
W20260923 23:14:52.344695 513 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.227.193:43785 (error 108)
I20260923 23:14:52.346537 1120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:52.346339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.193:43785 (local address 127.0.0.1:57874)
0923 23:14:52.346458 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:52.346506 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.193:43785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":49,"threads_started":1}
W20260923 23:14:52.346689 513 catalog_manager.cc:4584] TS 2cd0dd070b8643bf966ede03f18a5de4 (127.24.227.193:43785): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 71ad1531269a4c1e97b8907597b4046c: Network error: Client connection negotiation failed: client connection to 127.24.227.193:43785: connect: Connection refused (error 111)
I20260923 23:14:52.350217 1120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:52.350093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.194:38277 (local address 127.0.0.1:35752)
0923 23:14:52.350159 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:52.350188 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.194:38277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260923 23:14:52.350214 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 879
I20260923 23:14:52.355587 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 495
2026-09-23T23:14:52Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (2317 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790205259943518-25487-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 23:14:52 dist-test-slave-754n krb5kdc[1123](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 23:14:52 dist-test-slave-754n krb5kdc[1123](info): set up 2 sockets
Sep 23 23:14:52 dist-test-slave-754n krb5kdc[1123](info): commencing operation
krb5kdc: starting...
W20260923 23:14:53.348407 25487 mini_kdc.cc:121] Time spent starting KDC: real 0.971s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790205259943518-25487-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790205259943518-25487-0/krb5kdc/test-user.keytab.
Sep 23 23:14:53 dist-test-slave-754n krb5kdc[1123](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205293, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T23:14:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T23:14:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.227.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.227.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.227.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab.
Entry for principal kudu/127.24.227.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab.
I20260923 23:14:53.452688 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790205259943518-25487-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790205259943518-25487-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790205259943518-25487-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.254:45449
--webserver_interface=127.24.227.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab
--principal=kudu/127.24.227.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:38399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.227.254:45449
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:14:53.529348 1136 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:14:53.529551 1136 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:14:53.529590 1136 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:14:53.530592 1136 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 23:14:53.530652 1136 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:14:53.530674 1136 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:14:53.530701 1136 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 23:14:53.530728 1136 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 23:14:53.532048 1136 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:38399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790205259943518-25487-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790205259943518-25487-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.227.254:45449
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.254:45449
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.227.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1136
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790205259943518-25487-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:14:53.532290 1136 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:14:53.532495 1136 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260923 23:14:53.536996 1136 server_base.cc:1034] running on GCE node
W20260923 23:14:53.537015 1144 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:53.537122 1141 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:53.537128 1142 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:14:53.537418 1136 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:14:53.537683 1136 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:14:53.538846 1136 hybrid_clock.cc:648] HybridClock initialized: now 1790205293538835 us; error 45 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-09-23T23:14:53Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1182 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 23:14:53 dist-test-slave-754n krb5kdc[1149](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 23:14:53 dist-test-slave-754n krb5kdc[1149](info): set up 2 sockets
Sep 23 23:14:53 dist-test-slave-754n krb5kdc[1149](info): commencing operation
krb5kdc: starting...
W20260923 23:14:54.544430 25487 mini_kdc.cc:121] Time spent starting KDC: real 0.985s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/krb5kdc/test-user.keytab.
Sep 23 23:14:54 dist-test-slave-754n krb5kdc[1149](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205294, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T23:14:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T23:14:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.227.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.227.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.227.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab.
Entry for principal kudu/127.24.227.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab.
I20260923 23:14:54.650728 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.254:34525
--webserver_interface=127.24.227.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab
--principal=kudu/127.24.227.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:43417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.227.254:34525
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:14:54.726182 1162 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:14:54.726413 1162 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:14:54.726447 1162 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:14:54.727610 1162 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 23:14:54.727675 1162 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:14:54.727699 1162 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:14:54.727722 1162 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 23:14:54.727751 1162 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 23:14:54.729221 1162 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:43417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.227.254:34525
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab
--principal=kudu/127.24.227.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.254:34525
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.227.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1162
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:14:54.729492 1162 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:14:54.729775 1162 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:14:54.734377 1170 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:54.734392 1168 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:54.734411 1167 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:14:54.734552 1162 server_base.cc:1034] running on GCE node
I20260923 23:14:54.734858 1162 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:14:54.735139 1162 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:14:54.736311 1162 hybrid_clock.cc:648] HybridClock initialized: now 1790205294736297 us; error 48 us; skew 500 ppm
Sep 23 23:14:54 dist-test-slave-754n krb5kdc[1149](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205294, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:14:54.738603 1162 init.cc:377] Logged in from keytab as kudu/127.24.227.254@KRBTEST.COM (short username kudu)
I20260923 23:14:54.738996 1162 webserver.cc:460] Webserver started at http://127.24.227.254:41713/ using document root <none> and password file <none>
I20260923 23:14:54.739199 1162 fs_manager.cc:359] Metadata directory not provided
I20260923 23:14:54.739274 1162 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:14:54.739423 1162 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:14:54.740495 1162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/master-0/data/instance:
uuid: "cce235f7d53f4c9695cfd5d8d7f53ac9"
format_stamp: "Formatted at 2026-09-23 23:14:54 on dist-test-slave-754n"
server_key: "aa4994ef681e1a091956328c234b3791"
server_key_iv: "726a0d560e4f222ce1a1f5d1ca82ece5"
server_key_version: "encryptionkey@0"
I20260923 23:14:54.740842 1162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/master-0/wal/instance:
uuid: "cce235f7d53f4c9695cfd5d8d7f53ac9"
format_stamp: "Formatted at 2026-09-23 23:14:54 on dist-test-slave-754n"
server_key: "aa4994ef681e1a091956328c234b3791"
server_key_iv: "726a0d560e4f222ce1a1f5d1ca82ece5"
server_key_version: "encryptionkey@0"
I20260923 23:14:54.742091 1162 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 23:14:54.742743 1177 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:54.742898 1162 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 23:14:54.742990 1162 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/master-0/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/master-0/wal
uuid: "cce235f7d53f4c9695cfd5d8d7f53ac9"
format_stamp: "Formatted at 2026-09-23 23:14:54 on dist-test-slave-754n"
server_key: "aa4994ef681e1a091956328c234b3791"
server_key_iv: "726a0d560e4f222ce1a1f5d1ca82ece5"
server_key_version: "encryptionkey@0"
I20260923 23:14:54.743064 1162 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:14:54.773520 1162 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:14:54.775759 1162 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:14:54.775986 1162 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:14:54.779584 1162 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.254:34525
I20260923 23:14:54.779618 1229 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.254:34525 every 8 connection(s)
I20260923 23:14:54.779965 1162 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
I20260923 23:14:54.780817 1230 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:14:54.782894 1230 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cce235f7d53f4c9695cfd5d8d7f53ac9: Bootstrap starting.
I20260923 23:14:54.783439 1230 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cce235f7d53f4c9695cfd5d8d7f53ac9: Neither blocks nor log segments found. Creating new log.
I20260923 23:14:54.783710 1230 log.cc:824] T 00000000000000000000000000000000 P cce235f7d53f4c9695cfd5d8d7f53ac9: Log is configured to *not* fsync() on all Append() calls
I20260923 23:14:54.784409 1230 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cce235f7d53f4c9695cfd5d8d7f53ac9: No bootstrap required, opened a new log
I20260923 23:14:54.785020 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 1162
I20260923 23:14:54.785140 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/master-0/wal/instance
I20260923 23:14:54.785332 25487 external_mini_cluster.cc:1428] Setting key 8063bec542343023337c18a609611dbb
I20260923 23:14:54.786952 1230 raft_consensus.cc:348] T 00000000000000000000000000000000 P cce235f7d53f4c9695cfd5d8d7f53ac9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cce235f7d53f4c9695cfd5d8d7f53ac9" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 34525 } }
I20260923 23:14:54.787149 1230 raft_consensus.cc:374] T 00000000000000000000000000000000 P cce235f7d53f4c9695cfd5d8d7f53ac9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:14:54.787190 1230 raft_consensus.cc:729] T 00000000000000000000000000000000 P cce235f7d53f4c9695cfd5d8d7f53ac9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cce235f7d53f4c9695cfd5d8d7f53ac9, State: Initialized, Role: FOLLOWER
I20260923 23:14:54.787322 1230 consensus_queue.cc:260] T 00000000000000000000000000000000 P cce235f7d53f4c9695cfd5d8d7f53ac9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cce235f7d53f4c9695cfd5d8d7f53ac9" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 34525 } }
I20260923 23:14:54.787412 1230 raft_consensus.cc:388] T 00000000000000000000000000000000 P cce235f7d53f4c9695cfd5d8d7f53ac9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 23:14:54.787446 1230 raft_consensus.cc:482] T 00000000000000000000000000000000 P cce235f7d53f4c9695cfd5d8d7f53ac9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 23:14:54.787501 1230 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cce235f7d53f4c9695cfd5d8d7f53ac9 [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:14:54.788142 1230 raft_consensus.cc:504] T 00000000000000000000000000000000 P cce235f7d53f4c9695cfd5d8d7f53ac9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cce235f7d53f4c9695cfd5d8d7f53ac9" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 34525 } }
I20260923 23:14:54.788225 1230 leader_election.cc:302] T 00000000000000000000000000000000 P cce235f7d53f4c9695cfd5d8d7f53ac9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cce235f7d53f4c9695cfd5d8d7f53ac9; no voters:
I20260923 23:14:54.788465 1230 leader_election.cc:288] T 00000000000000000000000000000000 P cce235f7d53f4c9695cfd5d8d7f53ac9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 23:14:54.788492 1235 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cce235f7d53f4c9695cfd5d8d7f53ac9 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 23:14:54.788590 1235 raft_consensus.cc:686] T 00000000000000000000000000000000 P cce235f7d53f4c9695cfd5d8d7f53ac9 [term 1 LEADER]: Becoming Leader. State: Replica: cce235f7d53f4c9695cfd5d8d7f53ac9, State: Running, Role: LEADER
I20260923 23:14:54.788674 1235 consensus_queue.cc:237] T 00000000000000000000000000000000 P cce235f7d53f4c9695cfd5d8d7f53ac9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cce235f7d53f4c9695cfd5d8d7f53ac9" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 34525 } }
I20260923 23:14:54.788708 1230 sys_catalog.cc:564] T 00000000000000000000000000000000 P cce235f7d53f4c9695cfd5d8d7f53ac9 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 23:14:54.788997 1235 sys_catalog.cc:455] T 00000000000000000000000000000000 P cce235f7d53f4c9695cfd5d8d7f53ac9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cce235f7d53f4c9695cfd5d8d7f53ac9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cce235f7d53f4c9695cfd5d8d7f53ac9" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 34525 } } }
I20260923 23:14:54.789072 1235 sys_catalog.cc:458] T 00000000000000000000000000000000 P cce235f7d53f4c9695cfd5d8d7f53ac9 [sys.catalog]: This master's current role is: LEADER
I20260923 23:14:54.789196 1235 sys_catalog.cc:455] T 00000000000000000000000000000000 P cce235f7d53f4c9695cfd5d8d7f53ac9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cce235f7d53f4c9695cfd5d8d7f53ac9. Latest consensus state: current_term: 1 leader_uuid: "cce235f7d53f4c9695cfd5d8d7f53ac9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cce235f7d53f4c9695cfd5d8d7f53ac9" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 34525 } } }
I20260923 23:14:54.789261 1235 sys_catalog.cc:458] T 00000000000000000000000000000000 P cce235f7d53f4c9695cfd5d8d7f53ac9 [sys.catalog]: This master's current role is: LEADER
I20260923 23:14:54.789877 1241 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 23:14:54.790370 1241 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 23:14:54.791939 1241 catalog_manager.cc:1347] Generated new cluster ID: 2efc077ec3ac4d85b5d5a788865b2c9b
I20260923 23:14:54.792004 1241 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 23:14:54 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205294, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
I20260923 23:14:54.796552 1234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:54.785799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48370 (local address 127.24.227.254:34525)
0923 23:14:54.785963 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:54.785970 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:54.785992 (+ 22us) server_negotiation.cc:407] Connection header received
0923 23:14:54.786892 (+ 900us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:54.786899 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:54.786944 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:54.787068 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:54.787845 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:54.789048 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:54.790060 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:54.790509 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:54.790632 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:54.793104 (+ 2472us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:54.793107 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:54.793108 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:54.793117 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:54.794816 (+ 1699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:54.795201 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:54.795203 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:54.795204 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:54.795295 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:54.795506 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:54.795507 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:54.795507 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:54.795747 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:54.796217 (+ 470us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:54.796275 (+ 58us) server_negotiation.cc:299] Negotiation successful
0923 23:14:54.796386 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":85,"threads_started":1}
I20260923 23:14:54.798906 1241 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 23:14:54.799364 1241 catalog_manager.cc:1504] Loading token signing keys...
I20260923 23:14:54.807118 1241 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cce235f7d53f4c9695cfd5d8d7f53ac9: Generated new TSK 0
I20260923 23:14:54.807283 1241 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.227.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.227.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.227.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.193.keytab.
Entry for principal kudu/127.24.227.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.193.keytab.
I20260923 23:14:54.832799 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.227.193:0
--local_ip_for_outbound_sockets=127.24.227.193
--webserver_interface=127.24.227.193
--webserver_port=0
--tserver_master_addrs=127.24.227.254:34525
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.193.keytab
--principal=kudu/127.24.227.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:43417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:14:54.929247 1256 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:14:54.929476 1256 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:14:54.929515 1256 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:14:54.930637 1256 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:14:54.930701 1256 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:14:54.930743 1256 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.227.193
I20260923 23:14:54.932224 1256 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:43417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.193.keytab
--principal=kudu/127.24.227.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.227.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.227.193
--webserver_port=0
--tserver_master_addrs=127.24.227.254:34525
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1256
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.227.193
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:14:54.932471 1256 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:14:54.932685 1256 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:14:54.937717 1261 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:54.937799 1262 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:54.937733 1264 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:14:54.938051 1256 server_base.cc:1034] running on GCE node
I20260923 23:14:54.938254 1256 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:14:54.938483 1256 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:14:54.939633 1256 hybrid_clock.cc:648] HybridClock initialized: now 1790205294939621 us; error 28 us; skew 500 ppm
Sep 23 23:14:54 dist-test-slave-754n krb5kdc[1149](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205294, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:14:54.941998 1256 init.cc:377] Logged in from keytab as kudu/127.24.227.193@KRBTEST.COM (short username kudu)
I20260923 23:14:54.942483 1256 webserver.cc:460] Webserver started at http://127.24.227.193:38169/ using document root <none> and password file <none>
I20260923 23:14:54.942695 1256 fs_manager.cc:359] Metadata directory not provided
I20260923 23:14:54.942759 1256 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:14:54.942905 1256 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:14:54.943830 1256 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/ts-0/data/instance:
uuid: "5c7ed0436b4a41cc855f96a238e47e7a"
format_stamp: "Formatted at 2026-09-23 23:14:54 on dist-test-slave-754n"
server_key: "83eda11d88ec74c07ab4f35c22eb3f8d"
server_key_iv: "a6aaa5e322d474cbadcb5dcb854a159c"
server_key_version: "encryptionkey@0"
I20260923 23:14:54.944228 1256 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/ts-0/wal/instance:
uuid: "5c7ed0436b4a41cc855f96a238e47e7a"
format_stamp: "Formatted at 2026-09-23 23:14:54 on dist-test-slave-754n"
server_key: "83eda11d88ec74c07ab4f35c22eb3f8d"
server_key_iv: "a6aaa5e322d474cbadcb5dcb854a159c"
server_key_version: "encryptionkey@0"
I20260923 23:14:54.945497 1256 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 23:14:54.946231 1271 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:54.946409 1256 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 23:14:54.946504 1256 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/ts-0/wal
uuid: "5c7ed0436b4a41cc855f96a238e47e7a"
format_stamp: "Formatted at 2026-09-23 23:14:54 on dist-test-slave-754n"
server_key: "83eda11d88ec74c07ab4f35c22eb3f8d"
server_key_iv: "a6aaa5e322d474cbadcb5dcb854a159c"
server_key_version: "encryptionkey@0"
I20260923 23:14:54.946588 1256 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:14:54.958086 1256 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:14:54.959733 1256 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:14:54.959839 1256 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:14:54.960086 1256 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 23:14:54.960314 1256 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 23:14:54.960342 1256 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:54.960369 1256 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 23:14:54.960390 1256 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:54.965951 1256 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.193:43491
I20260923 23:14:54.965996 1384 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.193:43491 every 8 connection(s)
I20260923 23:14:54.966291 1256 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/ts-0/data/info.pb
I20260923 23:14:54.967464 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 1256
I20260923 23:14:54.967586 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/ts-0/wal/instance
I20260923 23:14:54.967799 25487 external_mini_cluster.cc:1428] Setting key a9c78b37a2c65eea509ed97608c115a7
Sep 23 23:14:54 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205294, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.193@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.24.227.194@KRBTEST.COM; defaulting to no policy
I20260923 23:14:54.976162 1234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:54.966930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.193:58079 (local address 127.24.227.254:34525)
0923 23:14:54.966987 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:54.966989 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:54.967162 (+ 173us) server_negotiation.cc:407] Connection header received
0923 23:14:54.967811 (+ 649us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:54.967812 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:54.967821 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:54.967939 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:54.969094 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:54.969586 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:54.970595 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:54.970799 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:54.970861 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:54.973109 (+ 2248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:54.973110 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:54.973111 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:54.973119 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:54.974549 (+ 1430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:54.974955 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:54.974956 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:54.974956 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:54.975016 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:54.975267 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:54.975268 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:54.975268 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:54.975524 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:54.975637 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:54.975986 (+ 349us) server_negotiation.cc:299] Negotiation successful
0923 23:14:54.976073 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260923 23:14:54.976380 1387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:54.966886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:34525 (local address 127.24.227.193:58079)
0923 23:14:54.967037 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:54.967076 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:54.967703 (+ 627us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:54.968091 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:54.968095 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:54.968726 (+ 631us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:14:54.968984 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:54.968991 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:54.969685 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:54.969686 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:54.970488 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:54.970491 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:54.970890 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:54.970891 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:54.971031 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:54.971471 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:14:54.971480 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:14:54.973014 (+ 1534us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:14:54.974651 (+ 1637us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:54.974657 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:54.974659 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:54.974878 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:54.975083 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:54.975084 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:54.975085 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:54.975192 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:54.975629 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:14:54.975636 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:14:54.975794 (+ 158us) client_negotiation.cc:769] Sending connection context
0923 23:14:54.975960 (+ 166us) client_negotiation.cc:240] Negotiation successful
0923 23:14:54.976112 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":87,"threads_started":1}
I20260923 23:14:54.976842 1385 heartbeater.cc:347] Connected to a master server at 127.24.227.254:34525
I20260923 23:14:54.976935 1385 heartbeater.cc:464] Registering TS with master...
I20260923 23:14:54.977101 1385 heartbeater.cc:511] Master 127.24.227.254:34525 requested a full tablet report, sending...
I20260923 23:14:54.977824 1194 ts_manager.cc:194] Registered new tserver with Master: 5c7ed0436b4a41cc855f96a238e47e7a (127.24.227.193:43491)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.227.194@KRBTEST.COM" created.
I20260923 23:14:54.978668 1194 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.227.193@KRBTEST.COM'} at 127.24.227.193:58079
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.227.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.194.keytab.
Entry for principal kudu/127.24.227.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.194.keytab.
I20260923 23:14:54.992157 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.227.194:0
--local_ip_for_outbound_sockets=127.24.227.194
--webserver_interface=127.24.227.194
--webserver_port=0
--tserver_master_addrs=127.24.227.254:34525
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.194.keytab
--principal=kudu/127.24.227.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:43417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:14:55.069705 1390 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:14:55.069880 1390 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:14:55.069900 1390 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:14:55.070969 1390 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:14:55.071007 1390 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:14:55.071030 1390 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.227.194
I20260923 23:14:55.072407 1390 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:43417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.194.keytab
--principal=kudu/127.24.227.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.227.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.227.194
--webserver_port=0
--tserver_master_addrs=127.24.227.254:34525
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1390
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.227.194
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:14:55.072623 1390 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:14:55.072829 1390 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:14:55.077462 1396 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:55.077479 1398 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:14:55.077529 1390 server_base.cc:1034] running on GCE node
W20260923 23:14:55.077487 1395 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:14:55.077924 1390 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:14:55.078154 1390 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:14:55.079330 1390 hybrid_clock.cc:648] HybridClock initialized: now 1790205295079305 us; error 36 us; skew 500 ppm
Sep 23 23:14:55 dist-test-slave-754n krb5kdc[1149](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205295, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:14:55.081630 1390 init.cc:377] Logged in from keytab as kudu/127.24.227.194@KRBTEST.COM (short username kudu)
I20260923 23:14:55.082012 1390 webserver.cc:460] Webserver started at http://127.24.227.194:46751/ using document root <none> and password file <none>
I20260923 23:14:55.082217 1390 fs_manager.cc:359] Metadata directory not provided
I20260923 23:14:55.082281 1390 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:14:55.082454 1390 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:14:55.083411 1390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/ts-1/data/instance:
uuid: "d8eb2d3eaee343ee82404d756a74e1a9"
format_stamp: "Formatted at 2026-09-23 23:14:55 on dist-test-slave-754n"
server_key: "cdf218beea7c0428d42d32a922d964d4"
server_key_iv: "ad5734e030a01a8a82ae17bc8977437e"
server_key_version: "encryptionkey@0"
I20260923 23:14:55.083772 1390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/ts-1/wal/instance:
uuid: "d8eb2d3eaee343ee82404d756a74e1a9"
format_stamp: "Formatted at 2026-09-23 23:14:55 on dist-test-slave-754n"
server_key: "cdf218beea7c0428d42d32a922d964d4"
server_key_iv: "ad5734e030a01a8a82ae17bc8977437e"
server_key_version: "encryptionkey@0"
I20260923 23:14:55.085059 1390 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 23:14:55.085736 1405 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:55.085915 1390 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 23:14:55.086009 1390 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/ts-1/wal
uuid: "d8eb2d3eaee343ee82404d756a74e1a9"
format_stamp: "Formatted at 2026-09-23 23:14:55 on dist-test-slave-754n"
server_key: "cdf218beea7c0428d42d32a922d964d4"
server_key_iv: "ad5734e030a01a8a82ae17bc8977437e"
server_key_version: "encryptionkey@0"
I20260923 23:14:55.086086 1390 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:14:55.095490 1390 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:14:55.097157 1390 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:14:55.097306 1390 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:14:55.097506 1390 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 23:14:55.097775 1390 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 23:14:55.097827 1390 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:55.097865 1390 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 23:14:55.097903 1390 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:55.103217 1390 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.194:43121
I20260923 23:14:55.103287 1518 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.194:43121 every 8 connection(s)
I20260923 23:14:55.103575 1390 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/ts-1/data/info.pb
I20260923 23:14:55.106207 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 1390
I20260923 23:14:55.106315 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/ts-1/wal/instance
I20260923 23:14:55.106498 25487 external_mini_cluster.cc:1428] Setting key e7d83294c0562e02fe07188308f34efe
Sep 23 23:14:55 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205295, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.194@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
I20260923 23:14:55.113314 1234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:55.104248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.194:56429 (local address 127.24.227.254:34525)
0923 23:14:55.104303 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:55.104305 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:55.104505 (+ 200us) server_negotiation.cc:407] Connection header received
0923 23:14:55.104929 (+ 424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:55.104930 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:55.104939 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:55.104993 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:55.105836 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:55.106601 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:55.107670 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:55.108007 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:55.108081 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:55.110417 (+ 2336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:55.110418 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:55.110418 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:55.110424 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:55.111854 (+ 1430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:55.112379 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:55.112379 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:55.112380 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:55.112431 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:55.112709 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:55.112709 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:55.112710 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:55.112800 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:55.112844 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:55.113201 (+ 357us) server_negotiation.cc:299] Negotiation successful
0923 23:14:55.113255 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260923 23:14:55.113603 1521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:55.104197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:34525 (local address 127.24.227.194:56429)
0923 23:14:55.104358 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:55.104420 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:55.104851 (+ 431us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:55.105067 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:55.105070 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:55.105596 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:14:55.105765 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:55.105769 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:55.106699 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:55.106700 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:55.107579 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:55.107581 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:55.108085 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:55.108086 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:55.108183 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:55.108720 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:14:55.108730 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:14:55.110329 (+ 1599us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:14:55.111983 (+ 1654us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:55.111989 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:55.111992 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:55.112301 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:55.112496 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:55.112497 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:55.112498 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:55.112637 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:55.112870 (+ 233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:14:55.112876 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:14:55.113048 (+ 172us) client_negotiation.cc:769] Sending connection context
0923 23:14:55.113174 (+ 126us) client_negotiation.cc:240] Negotiation successful
0923 23:14:55.113354 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":81,"threads_started":1}
I20260923 23:14:55.114118 1519 heartbeater.cc:347] Connected to a master server at 127.24.227.254:34525
I20260923 23:14:55.114202 1519 heartbeater.cc:464] Registering TS with master...
I20260923 23:14:55.114364 1519 heartbeater.cc:511] Master 127.24.227.254:34525 requested a full tablet report, sending...
I20260923 23:14:55.114760 1194 ts_manager.cc:194] Registered new tserver with Master: d8eb2d3eaee343ee82404d756a74e1a9 (127.24.227.194:43121)
I20260923 23:14:55.115154 1194 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.227.194@KRBTEST.COM'} at 127.24.227.194:56429
WARNING: no policy specified for kudu/127.24.227.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.227.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.227.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.195.keytab.
Entry for principal kudu/127.24.227.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.195.keytab.
I20260923 23:14:55.130864 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.227.195:0
--local_ip_for_outbound_sockets=127.24.227.195
--webserver_interface=127.24.227.195
--webserver_port=0
--tserver_master_addrs=127.24.227.254:34525
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.195.keytab
--principal=kudu/127.24.227.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:43417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:14:55.210234 1524 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:14:55.210433 1524 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:14:55.210455 1524 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:14:55.211524 1524 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:14:55.211557 1524 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:14:55.211581 1524 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.227.195
I20260923 23:14:55.212884 1524 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:43417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.195.keytab
--principal=kudu/127.24.227.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.227.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.227.195
--webserver_port=0
--tserver_master_addrs=127.24.227.254:34525
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1524
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.227.195
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:14:55.213060 1524 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:14:55.213213 1524 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:14:55.218228 1532 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:55.218220 1529 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:55.218222 1530 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:14:55.218456 1524 server_base.cc:1034] running on GCE node
I20260923 23:14:55.218631 1524 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:14:55.218802 1524 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:14:55.219935 1524 hybrid_clock.cc:648] HybridClock initialized: now 1790205295219922 us; error 25 us; skew 500 ppm
Sep 23 23:14:55 dist-test-slave-754n krb5kdc[1149](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205295, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:14:55.222296 1524 init.cc:377] Logged in from keytab as kudu/127.24.227.195@KRBTEST.COM (short username kudu)
I20260923 23:14:55.222751 1524 webserver.cc:460] Webserver started at http://127.24.227.195:44267/ using document root <none> and password file <none>
I20260923 23:14:55.222983 1524 fs_manager.cc:359] Metadata directory not provided
I20260923 23:14:55.223057 1524 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:14:55.223188 1524 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:14:55.224231 1524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/ts-2/data/instance:
uuid: "79822685170c473c9a2886733f704e43"
format_stamp: "Formatted at 2026-09-23 23:14:55 on dist-test-slave-754n"
server_key: "a31307593763c8c0c6f038b97b37f4f4"
server_key_iv: "7ac6e78253c9b24e9ba5e82ae4819031"
server_key_version: "encryptionkey@0"
I20260923 23:14:55.224601 1524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/ts-2/wal/instance:
uuid: "79822685170c473c9a2886733f704e43"
format_stamp: "Formatted at 2026-09-23 23:14:55 on dist-test-slave-754n"
server_key: "a31307593763c8c0c6f038b97b37f4f4"
server_key_iv: "7ac6e78253c9b24e9ba5e82ae4819031"
server_key_version: "encryptionkey@0"
I20260923 23:14:55.225886 1524 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 23:14:55.226554 1539 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:55.226717 1524 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 23:14:55.226812 1524 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/ts-2/wal
uuid: "79822685170c473c9a2886733f704e43"
format_stamp: "Formatted at 2026-09-23 23:14:55 on dist-test-slave-754n"
server_key: "a31307593763c8c0c6f038b97b37f4f4"
server_key_iv: "7ac6e78253c9b24e9ba5e82ae4819031"
server_key_version: "encryptionkey@0"
I20260923 23:14:55.226887 1524 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:14:55.235316 1524 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:14:55.237512 1524 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:14:55.237670 1524 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:14:55.237879 1524 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 23:14:55.238166 1524 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 23:14:55.238220 1524 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:55.238251 1524 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 23:14:55.238279 1524 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:55.243795 1524 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.195:45209
I20260923 23:14:55.243835 1652 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.195:45209 every 8 connection(s)
I20260923 23:14:55.244122 1524 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/ts-2/data/info.pb
I20260923 23:14:55.244819 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 1524
I20260923 23:14:55.244947 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/ts-2/wal/instance
I20260923 23:14:55.245214 25487 external_mini_cluster.cc:1428] Setting key 89392d731d49e2eaecda1293511ddede
Sep 23 23:14:55 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205295, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.195@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
I20260923 23:14:55.253160 1234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:55.244804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.195:47259 (local address 127.24.227.254:34525)
0923 23:14:55.244870 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:55.244872 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:55.245067 (+ 195us) server_negotiation.cc:407] Connection header received
0923 23:14:55.245677 (+ 610us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:55.245678 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:55.245688 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:55.245729 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:55.246610 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:55.247074 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:55.248121 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:55.248333 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:55.248417 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:55.250477 (+ 2060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:55.250479 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:55.250479 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:55.250485 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:55.251850 (+ 1365us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:55.252295 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:55.252296 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:55.252296 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:55.252352 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:55.252591 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:55.252592 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:55.252592 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:55.252697 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:55.252767 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:55.253011 (+ 244us) server_negotiation.cc:299] Negotiation successful
0923 23:14:55.253080 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260923 23:14:55.253314 1655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:55.244773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:34525 (local address 127.24.227.195:47259)
0923 23:14:55.244935 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:55.244971 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:55.245586 (+ 615us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:55.245802 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:55.245805 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:55.246336 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:14:55.246518 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:55.246523 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:55.247191 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:55.247191 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:55.248025 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:55.248028 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:55.248430 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:55.248431 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:55.248554 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:55.249026 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:14:55.249033 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:14:55.250369 (+ 1336us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:14:55.252012 (+ 1643us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:55.252018 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:55.252020 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:55.252216 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:55.252419 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:14:55.252419 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:14:55.252420 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:14:55.252522 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:14:55.252758 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:14:55.252764 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:14:55.252887 (+ 123us) client_negotiation.cc:769] Sending connection context
0923 23:14:55.252980 (+ 93us) client_negotiation.cc:240] Negotiation successful
0923 23:14:55.253113 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":100,"threads_started":1}
I20260923 23:14:55.253772 1653 heartbeater.cc:347] Connected to a master server at 127.24.227.254:34525
I20260923 23:14:55.253870 1653 heartbeater.cc:464] Registering TS with master...
I20260923 23:14:55.254035 1653 heartbeater.cc:511] Master 127.24.227.254:34525 requested a full tablet report, sending...
I20260923 23:14:55.254400 1194 ts_manager.cc:194] Registered new tserver with Master: 79822685170c473c9a2886733f704e43 (127.24.227.195:45209)
I20260923 23:14:55.254772 1194 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.227.195@KRBTEST.COM'} at 127.24.227.195:47259
I20260923 23:14:55.256721 25487 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.24.227.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.227.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.227.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/krb5kdc/oryx_127.24.227.193.keytab.
Entry for principal oryx/127.24.227.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/krb5kdc/oryx_127.24.227.193.keytab.
WARNING: no policy specified for oryx/127.24.227.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.227.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.227.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/krb5kdc/oryx_127.24.227.194.keytab.
Entry for principal oryx/127.24.227.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/krb5kdc/oryx_127.24.227.194.keytab.
WARNING: no policy specified for oryx/127.24.227.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.227.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.227.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/krb5kdc/oryx_127.24.227.195.keytab.
Entry for principal oryx/127.24.227.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/krb5kdc/oryx_127.24.227.195.keytab.
I20260923 23:14:55.318112 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 1256
I20260923 23:14:55.322371 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 1390
I20260923 23:14:55.326282 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 1524
I20260923 23:14:55.330371 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 1162
I20260923 23:14:55.334774 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.254:34525
--webserver_interface=127.24.227.254
--webserver_port=41713
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab
--principal=kudu/127.24.227.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:43417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.227.254:34525
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:14:55.411841 1662 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:14:55.412106 1662 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:14:55.412141 1662 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:14:55.413233 1662 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 23:14:55.413295 1662 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:14:55.413321 1662 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:14:55.413350 1662 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 23:14:55.413379 1662 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 23:14:55.414803 1662 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:43417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.227.254:34525
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab
--principal=kudu/127.24.227.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.254:34525
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.227.254
--webserver_port=41713
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1662
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:14:55.415071 1662 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:14:55.415303 1662 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:14:55.419919 1670 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:55.419940 1668 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:55.419931 1667 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:14:55.419967 1662 server_base.cc:1034] running on GCE node
I20260923 23:14:55.420308 1662 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:14:55.420603 1662 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:14:55.421763 1662 hybrid_clock.cc:648] HybridClock initialized: now 1790205295421742 us; error 33 us; skew 500 ppm
Sep 23 23:14:55 dist-test-slave-754n krb5kdc[1149](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205295, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:14:55.424132 1662 init.cc:377] Logged in from keytab as kudu/127.24.227.254@KRBTEST.COM (short username kudu)
I20260923 23:14:55.424566 1662 webserver.cc:460] Webserver started at http://127.24.227.254:41713/ using document root <none> and password file <none>
I20260923 23:14:55.424791 1662 fs_manager.cc:359] Metadata directory not provided
I20260923 23:14:55.424865 1662 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:14:55.426160 1662 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 23:14:55.426798 1677 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:55.426966 1662 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 23:14:55.427048 1662 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/master-0/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/master-0/wal
uuid: "cce235f7d53f4c9695cfd5d8d7f53ac9"
format_stamp: "Formatted at 2026-09-23 23:14:54 on dist-test-slave-754n"
server_key: "aa4994ef681e1a091956328c234b3791"
server_key_iv: "726a0d560e4f222ce1a1f5d1ca82ece5"
server_key_version: "encryptionkey@0"
I20260923 23:14:55.427364 1662 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:14:55.442785 1662 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:14:55.444528 1662 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:14:55.444629 1662 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:14:55.449378 1662 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.254:34525
I20260923 23:14:55.449434 1729 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.254:34525 every 8 connection(s)
I20260923 23:14:55.449767 1662 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
I20260923 23:14:55.451129 1730 sys_catalog.cc:263] Verifying existing consensus state
I20260923 23:14:55.451777 1730 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cce235f7d53f4c9695cfd5d8d7f53ac9: Bootstrap starting.
I20260923 23:14:55.458941 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 1662
I20260923 23:14:55.459251 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.227.193:43491
--local_ip_for_outbound_sockets=127.24.227.193
--tserver_master_addrs=127.24.227.254:34525
--webserver_port=38169
--webserver_interface=127.24.227.193
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.193.keytab
--principal=kudu/127.24.227.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:43417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.24.227.193
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/krb5kdc/oryx_127.24.227.193.keytab with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 23:14:55.459569 1730 log.cc:824] T 00000000000000000000000000000000 P cce235f7d53f4c9695cfd5d8d7f53ac9: Log is configured to *not* fsync() on all Append() calls
I20260923 23:14:55.461227 1730 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cce235f7d53f4c9695cfd5d8d7f53ac9: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 23:14:55.461503 1730 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cce235f7d53f4c9695cfd5d8d7f53ac9: Bootstrap complete.
I20260923 23:14:55.463624 1730 raft_consensus.cc:348] T 00000000000000000000000000000000 P cce235f7d53f4c9695cfd5d8d7f53ac9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cce235f7d53f4c9695cfd5d8d7f53ac9" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 34525 } }
I20260923 23:14:55.463863 1730 raft_consensus.cc:729] T 00000000000000000000000000000000 P cce235f7d53f4c9695cfd5d8d7f53ac9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cce235f7d53f4c9695cfd5d8d7f53ac9, State: Initialized, Role: FOLLOWER
I20260923 23:14:55.464068 1730 consensus_queue.cc:260] T 00000000000000000000000000000000 P cce235f7d53f4c9695cfd5d8d7f53ac9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cce235f7d53f4c9695cfd5d8d7f53ac9" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 34525 } }
I20260923 23:14:55.464130 1730 raft_consensus.cc:388] T 00000000000000000000000000000000 P cce235f7d53f4c9695cfd5d8d7f53ac9 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 23:14:55.464185 1730 raft_consensus.cc:482] T 00000000000000000000000000000000 P cce235f7d53f4c9695cfd5d8d7f53ac9 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 23:14:55.464231 1730 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cce235f7d53f4c9695cfd5d8d7f53ac9 [term 1 FOLLOWER]: Advancing to term 2
I20260923 23:14:55.465201 1730 raft_consensus.cc:504] T 00000000000000000000000000000000 P cce235f7d53f4c9695cfd5d8d7f53ac9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cce235f7d53f4c9695cfd5d8d7f53ac9" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 34525 } }
I20260923 23:14:55.465282 1730 leader_election.cc:302] T 00000000000000000000000000000000 P cce235f7d53f4c9695cfd5d8d7f53ac9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cce235f7d53f4c9695cfd5d8d7f53ac9; no voters:
I20260923 23:14:55.465484 1730 leader_election.cc:288] T 00000000000000000000000000000000 P cce235f7d53f4c9695cfd5d8d7f53ac9 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260923 23:14:55.465516 1734 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cce235f7d53f4c9695cfd5d8d7f53ac9 [term 2 FOLLOWER]: Leader election won for term 2
I20260923 23:14:55.465621 1734 raft_consensus.cc:686] T 00000000000000000000000000000000 P cce235f7d53f4c9695cfd5d8d7f53ac9 [term 2 LEADER]: Becoming Leader. State: Replica: cce235f7d53f4c9695cfd5d8d7f53ac9, State: Running, Role: LEADER
I20260923 23:14:55.465701 1734 consensus_queue.cc:237] T 00000000000000000000000000000000 P cce235f7d53f4c9695cfd5d8d7f53ac9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cce235f7d53f4c9695cfd5d8d7f53ac9" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 34525 } }
I20260923 23:14:55.465732 1730 sys_catalog.cc:564] T 00000000000000000000000000000000 P cce235f7d53f4c9695cfd5d8d7f53ac9 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 23:14:55.465973 1734 sys_catalog.cc:455] T 00000000000000000000000000000000 P cce235f7d53f4c9695cfd5d8d7f53ac9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "cce235f7d53f4c9695cfd5d8d7f53ac9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cce235f7d53f4c9695cfd5d8d7f53ac9" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 34525 } } }
I20260923 23:14:55.466050 1734 sys_catalog.cc:458] T 00000000000000000000000000000000 P cce235f7d53f4c9695cfd5d8d7f53ac9 [sys.catalog]: This master's current role is: LEADER
I20260923 23:14:55.466162 1734 sys_catalog.cc:455] T 00000000000000000000000000000000 P cce235f7d53f4c9695cfd5d8d7f53ac9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cce235f7d53f4c9695cfd5d8d7f53ac9. Latest consensus state: current_term: 2 leader_uuid: "cce235f7d53f4c9695cfd5d8d7f53ac9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cce235f7d53f4c9695cfd5d8d7f53ac9" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 34525 } } }
I20260923 23:14:55.466221 1734 sys_catalog.cc:458] T 00000000000000000000000000000000 P cce235f7d53f4c9695cfd5d8d7f53ac9 [sys.catalog]: This master's current role is: LEADER
I20260923 23:14:55.467450 1750 catalog_manager.cc:1259] Loaded cluster ID: 2efc077ec3ac4d85b5d5a788865b2c9b
I20260923 23:14:55.467500 1750 catalog_manager.cc:1552] T 00000000000000000000000000000000 P cce235f7d53f4c9695cfd5d8d7f53ac9: loading cluster ID for follower catalog manager: success
I20260923 23:14:55.468657 1750 catalog_manager.cc:1574] T 00000000000000000000000000000000 P cce235f7d53f4c9695cfd5d8d7f53ac9: acquiring CA information for follower catalog manager: success
I20260923 23:14:55.468946 1750 catalog_manager.cc:1602] T 00000000000000000000000000000000 P cce235f7d53f4c9695cfd5d8d7f53ac9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260923 23:14:55.469023 1738 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 23:14:55.469137 1738 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 23:14:55.469193 1738 catalog_manager.cc:1259] Loaded cluster ID: 2efc077ec3ac4d85b5d5a788865b2c9b
I20260923 23:14:55.469220 1738 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 23:14:55.469383 1738 catalog_manager.cc:1504] Loading token signing keys...
I20260923 23:14:55.469473 1738 catalog_manager.cc:5879] T 00000000000000000000000000000000 P cce235f7d53f4c9695cfd5d8d7f53ac9: Loaded TSK: 0
I20260923 23:14:55.469594 1738 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260923 23:14:55.541706 1733 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:14:55.541922 1733 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:14:55.541960 1733 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:14:55.543016 1733 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:14:55.543077 1733 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:14:55.543118 1733 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.227.193
I20260923 23:14:55.544642 1733 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:43417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/krb5kdc/oryx_127.24.227.193.keytab
--principal=oryx/127.24.227.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.227.193:43491
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.227.193
--webserver_port=38169
--tserver_master_addrs=127.24.227.254:34525
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1733
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.227.193
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:14:55.544886 1733 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:14:55.545097 1733 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:14:55.549862 1759 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:55.549909 1756 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:55.550071 1757 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:14:55.550247 1733 server_base.cc:1034] running on GCE node
I20260923 23:14:55.550421 1733 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:14:55.550647 1733 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:14:55.551800 1733 hybrid_clock.cc:648] HybridClock initialized: now 1790205295551783 us; error 32 us; skew 500 ppm
Sep 23 23:14:55 dist-test-slave-754n krb5kdc[1149](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205295, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.227.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:14:55.554363 1733 init.cc:377] Logged in from keytab as oryx/127.24.227.193@KRBTEST.COM (short username oryx)
I20260923 23:14:55.554852 1733 webserver.cc:460] Webserver started at http://127.24.227.193:38169/ using document root <none> and password file <none>
I20260923 23:14:55.555090 1733 fs_manager.cc:359] Metadata directory not provided
I20260923 23:14:55.555160 1733 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:14:55.556799 1733 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 23:14:55.557528 1766 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:55.557773 1733 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 23:14:55.557864 1733 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/ts-0/wal
uuid: "5c7ed0436b4a41cc855f96a238e47e7a"
format_stamp: "Formatted at 2026-09-23 23:14:54 on dist-test-slave-754n"
server_key: "83eda11d88ec74c07ab4f35c22eb3f8d"
server_key_iv: "a6aaa5e322d474cbadcb5dcb854a159c"
server_key_version: "encryptionkey@0"
I20260923 23:14:55.558266 1733 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:14:55.586033 1733 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:14:55.588249 1733 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:14:55.588384 1733 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:14:55.588661 1733 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 23:14:55.588956 1733 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 23:14:55.589010 1733 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:55.589037 1733 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 23:14:55.589052 1733 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:55.595717 1733 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.193:43491
I20260923 23:14:55.595808 1879 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.193:43491 every 8 connection(s)
I20260923 23:14:55.596114 1733 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/ts-0/data/info.pb
Sep 23 23:14:55 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.193@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 23:14:55 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.193@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 23:14:55.604146 1882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:55.596896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:34525 (local address 127.24.227.193:38503)
0923 23:14:55.597071 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:55.597112 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:55.597698 (+ 586us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:55.598077 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:55.598081 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:55.598729 (+ 648us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:14:55.598942 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:55.598946 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:55.599912 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:55.599914 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:55.600851 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:55.600854 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:55.601351 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:55.601352 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:55.601479 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:55.601884 (+ 405us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:14:55.601893 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:14:55.603985 (+ 2092us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.227.254:34525: Server oryx/127.24.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":89,"threads_started":1}
W20260923 23:14:55.604206 1883 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 23:14:55.596932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.193:38503 (local address 127.24.227.254:34525)
0923 23:14:55.597105 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:55.597113 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:55.597187 (+ 74us) server_negotiation.cc:407] Connection header received
0923 23:14:55.597857 (+ 670us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:55.597862 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:55.597901 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:55.598013 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:55.599040 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:55.599796 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:55.600933 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:55.601269 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:55.601339 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:55.604099 (+ 2760us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.227.193:38503: BlockingRecv error: recv got EOF from 127.24.227.193:38503 (error 108)
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":73,"threads_started":1}
I20260923 23:14:55.604300 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 1733
I20260923 23:14:55.604632 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.227.194:43121
--local_ip_for_outbound_sockets=127.24.227.194
--tserver_master_addrs=127.24.227.254:34525
--webserver_port=46751
--webserver_interface=127.24.227.194
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.194.keytab
--principal=kudu/127.24.227.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:43417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.24.227.194
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/krb5kdc/oryx_127.24.227.194.keytab with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:14:55.605103 1880 heartbeater.cc:644] Failed to heartbeat to 127.24.227.254:34525 (0 consecutive failures): Not authorized: Failed to ping master at 127.24.227.254:34525: Client connection negotiation failed: client connection to 127.24.227.254:34525: Server oryx/127.24.227.254@KRBTEST.COM not found in Kerberos database
Sep 23 23:14:55 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.193@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 23:14:55 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.193@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 23:14:55.610109 1882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:55.605340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:34525 (local address 127.24.227.193:41051)
0923 23:14:55.605445 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:55.605457 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:55.605509 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:55.605627 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:55.605627 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:55.605791 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:14:55.605842 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:55.605844 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:55.606718 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:55.606718 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:55.607447 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:55.607450 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:55.607774 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:55.607775 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:55.607846 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:55.608209 (+ 363us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:14:55.608214 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:14:55.610038 (+ 1824us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.227.254:34525: Server oryx/127.24.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":88}
W20260923 23:14:55.610157 1883 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 23:14:55.605400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.193:41051 (local address 127.24.227.254:34525)
0923 23:14:55.605462 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:55.605463 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:55.605495 (+ 32us) server_negotiation.cc:407] Connection header received
0923 23:14:55.605566 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:55.605567 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:55.605577 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:55.605622 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:55.605915 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:55.606630 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:55.607528 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:55.607702 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:55.607754 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:55.610102 (+ 2348us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.227.193:41051: BlockingRecv error: recv got EOF from 127.24.227.193:41051 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Sep 23 23:14:55 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.193@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 23:14:55 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.193@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 23:14:55.614696 1882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:55.610424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:34525 (local address 127.24.227.193:51737)
0923 23:14:55.610477 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:55.610490 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:55.610534 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:55.610663 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:55.610664 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:55.610811 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:14:55.610844 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:55.610846 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:55.611392 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:55.611393 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:55.612293 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:55.612296 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:55.612624 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:55.612625 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:55.612686 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:55.613045 (+ 359us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:14:55.613051 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:14:55.614624 (+ 1573us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.227.254:34525: Server oryx/127.24.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":34}
W20260923 23:14:55.614742 1883 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 23:14:55.610488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.193:51737 (local address 127.24.227.254:34525)
0923 23:14:55.610534 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:55.610536 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:55.610544 (+ 8us) server_negotiation.cc:407] Connection header received
0923 23:14:55.610590 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:55.610591 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:55.610598 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:55.610641 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:55.610914 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:55.611320 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:55.612370 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:55.612535 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:55.612601 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:55.614687 (+ 2086us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.227.193:51737: BlockingRecv error: recv got EOF from 127.24.227.193:51737 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
W20260923 23:14:55.614795 1880 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260923 23:14:55.682562 1884 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:14:55.682751 1884 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:14:55.682785 1884 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:14:55.683779 1884 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:14:55.683838 1884 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:14:55.683876 1884 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.227.194
I20260923 23:14:55.685233 1884 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:43417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/krb5kdc/oryx_127.24.227.194.keytab
--principal=oryx/127.24.227.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.227.194:43121
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.227.194
--webserver_port=46751
--tserver_master_addrs=127.24.227.254:34525
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1884
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.227.194
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:14:55.685487 1884 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:14:55.685719 1884 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:14:55.690231 1889 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:55.690218 1890 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:14:55.690313 1884 server_base.cc:1034] running on GCE node
W20260923 23:14:55.690244 1892 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:14:55.690611 1884 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:14:55.690773 1884 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:14:55.691937 1884 hybrid_clock.cc:648] HybridClock initialized: now 1790205295691914 us; error 37 us; skew 500 ppm
Sep 23 23:14:55 dist-test-slave-754n krb5kdc[1149](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205295, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.227.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:14:55.694082 1884 init.cc:377] Logged in from keytab as oryx/127.24.227.194@KRBTEST.COM (short username oryx)
I20260923 23:14:55.694453 1884 webserver.cc:460] Webserver started at http://127.24.227.194:46751/ using document root <none> and password file <none>
I20260923 23:14:55.694664 1884 fs_manager.cc:359] Metadata directory not provided
I20260923 23:14:55.694732 1884 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:14:55.696004 1884 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 23:14:55.696621 1899 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:55.696781 1884 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 23:14:55.696856 1884 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/ts-1/wal
uuid: "d8eb2d3eaee343ee82404d756a74e1a9"
format_stamp: "Formatted at 2026-09-23 23:14:55 on dist-test-slave-754n"
server_key: "cdf218beea7c0428d42d32a922d964d4"
server_key_iv: "ad5734e030a01a8a82ae17bc8977437e"
server_key_version: "encryptionkey@0"
I20260923 23:14:55.697117 1884 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:14:55.720216 1884 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:14:55.722330 1884 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:14:55.722431 1884 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:14:55.722585 1884 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 23:14:55.722807 1884 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 23:14:55.722833 1884 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:55.722851 1884 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 23:14:55.722863 1884 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:55.728355 1884 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.194:43121
I20260923 23:14:55.728433 2012 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.194:43121 every 8 connection(s)
I20260923 23:14:55.728667 1884 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/ts-1/data/info.pb
I20260923 23:14:55.729971 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 1884
I20260923 23:14:55.730322 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.227.195:45209
--local_ip_for_outbound_sockets=127.24.227.195
--tserver_master_addrs=127.24.227.254:34525
--webserver_port=44267
--webserver_interface=127.24.227.195
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.195.keytab
--principal=kudu/127.24.227.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:43417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.24.227.195
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/krb5kdc/oryx_127.24.227.195.keytab with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 23 23:14:55 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.194@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 23:14:55 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.194@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 23:14:55.736366 2015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:55.729258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:34525 (local address 127.24.227.194:56863)
0923 23:14:55.729396 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:55.729432 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:55.729915 (+ 483us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:55.730243 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:55.730247 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:55.730955 (+ 708us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:14:55.731125 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:55.731129 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:55.732043 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:55.732045 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:55.733042 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:55.733044 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:55.733499 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:55.733499 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:55.733623 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:55.734064 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:14:55.734072 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:14:55.736244 (+ 2172us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.227.254:34525: Server oryx/127.24.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":77,"threads_started":1}
W20260923 23:14:55.736430 1883 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 23:14:55.729296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.194:56863 (local address 127.24.227.254:34525)
0923 23:14:55.729355 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:55.729357 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:55.729523 (+ 166us) server_negotiation.cc:407] Connection header received
0923 23:14:55.730014 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:55.730015 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:55.730032 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:55.730108 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:55.731215 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:55.731930 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:55.733124 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:55.733420 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:55.733491 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:55.736353 (+ 2862us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.227.194:56863: BlockingRecv error: recv got EOF from 127.24.227.194:56863 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260923 23:14:55.736613 2013 heartbeater.cc:644] Failed to heartbeat to 127.24.227.254:34525 (0 consecutive failures): Not authorized: Failed to ping master at 127.24.227.254:34525: Client connection negotiation failed: client connection to 127.24.227.254:34525: Server oryx/127.24.227.254@KRBTEST.COM not found in Kerberos database
Sep 23 23:14:55 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.194@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 23:14:55 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.194@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 23:14:55.740859 2015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:55.736806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:34525 (local address 127.24.227.194:57863)
0923 23:14:55.736841 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:55.736849 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:55.736904 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:55.737022 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:55.737023 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:55.737135 (+ 112us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:14:55.737170 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:55.737171 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:55.737806 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:55.737807 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:55.738293 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:55.738294 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:55.738671 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:55.738672 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:55.738735 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:55.739074 (+ 339us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:14:55.739078 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:14:55.740788 (+ 1710us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.227.254:34525: Server oryx/127.24.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":23}
W20260923 23:14:55.740939 1883 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 23:14:55.736861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.194:57863 (local address 127.24.227.254:34525)
0923 23:14:55.736925 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:55.736927 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:55.736937 (+ 10us) server_negotiation.cc:407] Connection header received
0923 23:14:55.736958 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:55.736958 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:55.736967 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:55.737011 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:55.737239 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:55.737711 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:55.738389 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:55.738598 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:55.738662 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:55.740870 (+ 2208us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.227.194:57863: BlockingRecv error: recv got EOF from 127.24.227.194:57863 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Sep 23 23:14:55 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.194@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 23:14:55 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.194@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 23:14:55.745090 2015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:55.741155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:34525 (local address 127.24.227.194:42765)
0923 23:14:55.741187 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:55.741195 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:55.741222 (+ 27us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:55.741315 (+ 93us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:55.741315 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:55.741433 (+ 118us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:14:55.741459 (+ 26us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:55.741460 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:55.741964 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:55.741965 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:55.742708 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:55.742710 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:55.743001 (+ 291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:55.743002 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:55.743054 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:55.743340 (+ 286us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:14:55.743345 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:14:55.745028 (+ 1683us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.227.254:34525: Server oryx/127.24.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":21}
W20260923 23:14:55.745191 2013 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260923 23:14:55.745146 1883 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 23:14:55.741207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.194:42765 (local address 127.24.227.254:34525)
0923 23:14:55.741241 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:55.741242 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:55.741249 (+ 7us) server_negotiation.cc:407] Connection header received
0923 23:14:55.741264 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:55.741264 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:55.741270 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:55.741311 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:55.741528 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:55.741900 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:55.742780 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:55.742939 (+ 159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:55.742985 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:55.745092 (+ 2107us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.227.194:42765: BlockingRecv error: recv got EOF from 127.24.227.194:42765 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260923 23:14:55.810253 2016 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:14:55.810413 2016 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:14:55.810431 2016 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:14:55.811384 2016 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:14:55.811414 2016 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:14:55.811435 2016 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.227.195
I20260923 23:14:55.812657 2016 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:43417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/krb5kdc/oryx_127.24.227.195.keytab
--principal=oryx/127.24.227.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.227.195:45209
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.227.195
--webserver_port=44267
--tserver_master_addrs=127.24.227.254:34525
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2016
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.227.195
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:14:55.812814 2016 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:14:55.812983 2016 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:14:55.817675 2022 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:55.817719 2021 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:14:55.817683 2024 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:14:55.817935 2016 server_base.cc:1034] running on GCE node
I20260923 23:14:55.818169 2016 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:14:55.818411 2016 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:14:55.819594 2016 hybrid_clock.cc:648] HybridClock initialized: now 1790205295819568 us; error 37 us; skew 500 ppm
Sep 23 23:14:55 dist-test-slave-754n krb5kdc[1149](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205295, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.227.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:14:55.822031 2016 init.cc:377] Logged in from keytab as oryx/127.24.227.195@KRBTEST.COM (short username oryx)
I20260923 23:14:55.822381 2016 webserver.cc:460] Webserver started at http://127.24.227.195:44267/ using document root <none> and password file <none>
I20260923 23:14:55.822574 2016 fs_manager.cc:359] Metadata directory not provided
I20260923 23:14:55.822620 2016 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:14:55.823747 2016 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 23:14:55.824301 2031 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:55.824448 2016 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:55.824569 2016 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/ts-2/wal
uuid: "79822685170c473c9a2886733f704e43"
format_stamp: "Formatted at 2026-09-23 23:14:55 on dist-test-slave-754n"
server_key: "a31307593763c8c0c6f038b97b37f4f4"
server_key_iv: "7ac6e78253c9b24e9ba5e82ae4819031"
server_key_version: "encryptionkey@0"
I20260923 23:14:55.824898 2016 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:14:55.833887 2016 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:14:55.835536 2016 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:14:55.835634 2016 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:14:55.835776 2016 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 23:14:55.836057 2016 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 23:14:55.836087 2016 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:55.836107 2016 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 23:14:55.836117 2016 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 23:14:55.841430 2016 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.195:45209
I20260923 23:14:55.841480 2144 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.195:45209 every 8 connection(s)
I20260923 23:14:55.841837 2016 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790205259943518-25487-0/minicluster-data/ts-2/data/info.pb
I20260923 23:14:55.845264 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 2016
Sep 23 23:14:55 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.195@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 23:14:55 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.195@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 23:14:55.849267 2147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:55.842461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:34525 (local address 127.24.227.195:45305)
0923 23:14:55.842626 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:55.842690 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:55.843152 (+ 462us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:55.843380 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:55.843383 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:55.843951 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:14:55.844131 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:55.844136 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:55.844829 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:55.844831 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:55.845824 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:55.845827 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:55.846326 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:55.846327 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:55.846426 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:55.846873 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:14:55.846880 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:14:55.849133 (+ 2253us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.227.254:34525: Server oryx/127.24.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":80,"threads_started":1}
W20260923 23:14:55.849306 1883 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 23:14:55.842485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.195:45305 (local address 127.24.227.254:34525)
0923 23:14:55.842566 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:55.842568 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:55.842778 (+ 210us) server_negotiation.cc:407] Connection header received
0923 23:14:55.843252 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:55.843253 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:55.843265 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:55.843316 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:55.844233 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:55.844737 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:55.845929 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:55.846244 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:55.846319 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:55.849242 (+ 2923us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.227.195:45305: BlockingRecv error: recv got EOF from 127.24.227.195:45305 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
W20260923 23:14:55.849507 2145 heartbeater.cc:644] Failed to heartbeat to 127.24.227.254:34525 (0 consecutive failures): Not authorized: Failed to ping master at 127.24.227.254:34525: Client connection negotiation failed: client connection to 127.24.227.254:34525: Server oryx/127.24.227.254@KRBTEST.COM not found in Kerberos database
Sep 23 23:14:55 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.195@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 23:14:55.853152 2149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:55.845761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48376 (local address 127.24.227.254:34525)
0923 23:14:55.845895 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:55.845897 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:55.845906 (+ 9us) server_negotiation.cc:407] Connection header received
0923 23:14:55.845986 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:55.845987 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:55.845996 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:55.846046 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:55.846687 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:55.847250 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:55.848244 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:55.848476 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:55.848568 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:55.850054 (+ 1486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:14:55.850057 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:14:55.850058 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:14:55.850068 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:14:55.851701 (+ 1633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:55.852112 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:55.852115 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:55.852116 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:55.852231 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:14:55.852449 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:14:55.852450 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:14:55.852450 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:14:55.852643 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:14:55.852733 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0923 23:14:55.852936 (+ 203us) server_negotiation.cc:299] Negotiation successful
0923 23:14:55.853045 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":70,"threads_started":1}
Sep 23 23:14:55 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.195@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 23:14:55.854167 2147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:55.849728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:34525 (local address 127.24.227.195:47279)
0923 23:14:55.849777 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:55.849786 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:55.849827 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:55.849953 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:55.849954 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:55.850058 (+ 104us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:14:55.850094 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:55.850096 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:55.850952 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:55.850953 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:55.851441 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:55.851443 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:55.851902 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:55.851903 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:55.851969 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:55.852375 (+ 406us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:14:55.852380 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:14:55.854129 (+ 1749us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.227.254:34525: Server oryx/127.24.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":34}
W20260923 23:14:55.854251 1883 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 23:14:55.849798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.195:47279 (local address 127.24.227.254:34525)
0923 23:14:55.849830 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:55.849832 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:55.849842 (+ 10us) server_negotiation.cc:407] Connection header received
0923 23:14:55.849880 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:55.849880 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:55.849888 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:55.849946 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:55.850163 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:55.850866 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:55.851520 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:55.851815 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:55.851909 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:55.854197 (+ 2288us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.227.195:47279: BlockingRecv error: recv got EOF from 127.24.227.195:47279 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
Sep 23 23:14:55 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.195@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 23:14:55 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.195@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 23:14:55.857755 2147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:55.854411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:34525 (local address 127.24.227.195:55543)
0923 23:14:55.854502 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:55.854514 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:55.854550 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:55.854644 (+ 94us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:55.854644 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:55.854739 (+ 95us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:14:55.854759 (+ 20us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:55.854760 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:55.855268 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:55.855268 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:55.855673 (+ 405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:55.855674 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:55.855992 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:55.855993 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:55.856023 (+ 30us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:55.856330 (+ 307us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:14:55.856334 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:14:55.857717 (+ 1383us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.227.254:34525: Server oryx/127.24.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":77}
W20260923 23:14:55.857853 1883 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 23:14:55.854464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.195:55543 (local address 127.24.227.254:34525)
0923 23:14:55.854500 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:55.854502 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:55.854542 (+ 40us) server_negotiation.cc:407] Connection header received
0923 23:14:55.854584 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:55.854584 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:55.854591 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:55.854640 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:55.854811 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:55.855198 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:55.855731 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:55.855930 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:55.855990 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:55.857793 (+ 1803us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.227.195:55543: BlockingRecv error: recv got EOF from 127.24.227.195:55543 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260923 23:14:55.857892 2145 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Sep 23 23:14:56 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.193@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 23:14:56 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.193@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 23:14:56.625258 2150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:56.615113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:34525 (local address 127.24.227.193:38883)
0923 23:14:56.615231 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:56.615242 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:56.615293 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:56.615408 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:56.615408 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:56.615549 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:14:56.615596 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:56.615598 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:56.616267 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:56.616268 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:56.616786 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:56.616787 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:56.617090 (+ 303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:56.617091 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:56.617129 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:56.623269 (+ 6140us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:14:56.623279 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:14:56.625160 (+ 1881us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.227.254:34525: Server oryx/127.24.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":73,"threads_started":1}
W20260923 23:14:56.625322 2151 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 23:14:56.615175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.193:38883 (local address 127.24.227.254:34525)
0923 23:14:56.615299 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:56.615301 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:56.615310 (+ 9us) server_negotiation.cc:407] Connection header received
0923 23:14:56.615331 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:56.615332 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:56.615341 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:56.615397 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:56.615653 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:56.616188 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:56.616853 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:56.617030 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:56.617078 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:56.625251 (+ 8173us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.227.193:38883: BlockingRecv error: recv got EOF from 127.24.227.193:38883 (error 108)
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":72,"threads_started":1}
Sep 23 23:14:56 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.194@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 23:14:56 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.194@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 23:14:56.750159 2152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:56.745529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:34525 (local address 127.24.227.194:54695)
0923 23:14:56.745694 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:56.745705 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:56.745777 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:56.745948 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:56.745948 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:56.746112 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:14:56.746156 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:56.746158 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:56.746806 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:56.746808 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:56.747336 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:56.747338 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:56.747775 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:56.747776 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:56.747836 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:56.748297 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:14:56.748301 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:14:56.750103 (+ 1802us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.227.254:34525: Server oryx/127.24.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":83,"threads_started":1}
W20260923 23:14:56.750272 2151 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 23:14:56.745616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.194:54695 (local address 127.24.227.254:34525)
0923 23:14:56.745694 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:56.745697 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:56.745760 (+ 63us) server_negotiation.cc:407] Connection header received
0923 23:14:56.745831 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:56.745832 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:56.745845 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:56.745942 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:56.746247 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:56.746733 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:56.747431 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:56.747700 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:56.747777 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:56.750197 (+ 2420us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.227.194:54695: BlockingRecv error: recv got EOF from 127.24.227.194:54695 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Sep 23 23:14:56 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.195@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 23:14:56 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.195@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 23:14:56.862546 2153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:56.858248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:34525 (local address 127.24.227.195:33855)
0923 23:14:56.858435 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:56.858444 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:56.858498 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:56.858622 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:56.858622 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:56.858783 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:14:56.858843 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:56.858845 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:56.859400 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:56.859401 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:56.859942 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:56.859944 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:56.860391 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:56.860392 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:56.860461 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:56.860896 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:14:56.860901 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:14:56.862500 (+ 1599us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.227.254:34525: Server oryx/127.24.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":109,"threads_started":1}
W20260923 23:14:56.862638 2151 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 23:14:56.858340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.195:33855 (local address 127.24.227.254:34525)
0923 23:14:56.858395 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:56.858397 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:56.858484 (+ 87us) server_negotiation.cc:407] Connection header received
0923 23:14:56.858539 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:56.858540 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:56.858552 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:56.858602 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:56.858903 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:56.859341 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:56.860031 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:56.860319 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:56.860382 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:56.862576 (+ 2194us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.227.195:33855: BlockingRecv error: recv got EOF from 127.24.227.195:33855 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Sep 23 23:14:57 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.193@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 23:14:57 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.193@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 23:14:57.633751 2154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:57.625702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:34525 (local address 127.24.227.193:35359)
0923 23:14:57.625846 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:57.625858 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:57.625937 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:57.626078 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:57.626078 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:57.626252 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:14:57.626300 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:57.626302 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:57.626936 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:57.626937 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:57.627538 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:57.627541 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:57.627885 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:57.627886 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:57.627962 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:57.632156 (+ 4194us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:14:57.632161 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:14:57.633700 (+ 1539us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.227.254:34525: Server oryx/127.24.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":61,"threads_started":1}
W20260923 23:14:57.633893 2155 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 23:14:57.625746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.193:35359 (local address 127.24.227.254:34525)
0923 23:14:57.625892 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:57.625894 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:57.625921 (+ 27us) server_negotiation.cc:407] Connection header received
0923 23:14:57.625986 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:57.625986 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:57.625996 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:57.626063 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:57.626362 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:57.626854 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:57.627603 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:57.627791 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:57.627846 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:57.633777 (+ 5931us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.227.193:35359: BlockingRecv error: recv got EOF from 127.24.227.193:35359 (error 108)
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":76,"threads_started":1}
Sep 23 23:14:57 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.194@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 23:14:57 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.194@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 23:14:57.755016 2156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:57.750586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:34525 (local address 127.24.227.194:36221)
0923 23:14:57.750760 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:57.750772 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:57.750840 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:57.750967 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:57.750968 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:57.751132 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:14:57.751176 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:57.751178 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:57.751812 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:57.751813 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:57.752397 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:57.752399 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:57.752755 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:57.752756 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:57.752813 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:57.753294 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:14:57.753299 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:14:57.754959 (+ 1660us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.227.254:34525: Server oryx/127.24.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":99,"threads_started":1}
W20260923 23:14:57.755143 2155 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 23:14:57.750700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.194:36221 (local address 127.24.227.254:34525)
0923 23:14:57.750763 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:57.750766 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:57.750832 (+ 66us) server_negotiation.cc:407] Connection header received
0923 23:14:57.750890 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:57.750890 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:57.750902 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:57.750955 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:57.751251 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:57.751715 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:57.752492 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:57.752685 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:57.752758 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:57.755067 (+ 2309us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.227.194:36221: BlockingRecv error: recv got EOF from 127.24.227.194:36221 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Sep 23 23:14:57 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.195@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 23:14:57 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.195@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 23:14:57.867813 2157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:57.862971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:34525 (local address 127.24.227.195:56073)
0923 23:14:57.863095 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:57.863106 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:57.863183 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:57.863333 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:57.863333 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:57.863577 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:14:57.863634 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:57.863639 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:57.864387 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:57.864388 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:57.864979 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:57.864981 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:57.865399 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:57.865400 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:57.865469 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:57.865907 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:14:57.865912 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:14:57.867735 (+ 1823us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.227.254:34525: Server oryx/127.24.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":53,"threads_started":1}
W20260923 23:14:57.867978 2155 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 23:14:57.863135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.195:56073 (local address 127.24.227.254:34525)
0923 23:14:57.863209 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:57.863211 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:57.863220 (+ 9us) server_negotiation.cc:407] Connection header received
0923 23:14:57.863244 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:57.863245 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:57.863263 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:57.863326 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:57.863717 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:57.864303 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:57.865102 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:57.865311 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:57.865398 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:57.867867 (+ 2469us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.227.195:56073: BlockingRecv error: recv got EOF from 127.24.227.195:56073 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Sep 23 23:14:58 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.193@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 23:14:58 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.193@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 23:14:58.638690 2158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:58.634180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:34525 (local address 127.24.227.193:43679)
0923 23:14:58.634326 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:58.634336 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:58.634394 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:58.634515 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:58.634515 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:58.634662 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:14:58.634708 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:58.634709 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:58.635334 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:58.635335 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:58.635839 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:58.635840 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:58.636318 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:58.636319 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:58.636361 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:58.636769 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:14:58.636774 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:14:58.638582 (+ 1808us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.227.254:34525: Server oryx/127.24.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":87,"threads_started":1}
W20260923 23:14:58.638746 2159 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 23:14:58.634247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.193:43679 (local address 127.24.227.254:34525)
0923 23:14:58.634340 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:58.634342 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:58.634395 (+ 53us) server_negotiation.cc:407] Connection header received
0923 23:14:58.634436 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:58.634437 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:58.634447 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:58.634503 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:58.634768 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:58.635271 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:58.635991 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:58.636255 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:58.636307 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:58.638670 (+ 2363us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.227.193:43679: BlockingRecv error: recv got EOF from 127.24.227.193:43679 (error 108)
Metrics: {"server-negotiator.queue_time_us":73,"thread_start_us":46,"threads_started":1}
Sep 23 23:14:58 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.194@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 23:14:58 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.194@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 23:14:58.759907 2160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:58.755478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:34525 (local address 127.24.227.194:42477)
0923 23:14:58.755667 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:58.755687 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:58.755742 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:58.755897 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:58.755898 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:58.756061 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:14:58.756109 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:58.756110 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:58.756719 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:58.756719 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:58.757221 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:58.757222 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:58.757531 (+ 309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:58.757532 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:58.757568 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:58.758007 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:14:58.758011 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:14:58.759779 (+ 1768us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.227.254:34525: Server oryx/127.24.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":83,"threads_started":1}
W20260923 23:14:58.759944 2159 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 23:14:58.755688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.194:42477 (local address 127.24.227.254:34525)
0923 23:14:58.755748 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:58.755750 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:58.755785 (+ 35us) server_negotiation.cc:407] Connection header received
0923 23:14:58.755807 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:58.755808 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:58.755818 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:58.755856 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:58.756169 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:58.756651 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:58.757291 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:58.757470 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:58.757523 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:58.759859 (+ 2336us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.227.194:42477: BlockingRecv error: recv got EOF from 127.24.227.194:42477 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Sep 23 23:14:58 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.195@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 23:14:58 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.195@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 23:14:58.872745 2161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:58.868467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:34525 (local address 127.24.227.195:40645)
0923 23:14:58.868696 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:58.868708 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:58.868774 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:58.868928 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:58.868929 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:58.869083 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:14:58.869131 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:58.869132 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:58.869760 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:58.869760 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:58.870275 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:58.870276 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:58.870601 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:58.870602 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:58.870638 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:58.871054 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:14:58.871058 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:14:58.872680 (+ 1622us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.227.254:34525: Server oryx/127.24.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":114,"threads_started":1}
W20260923 23:14:58.872884 2159 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 23:14:58.868555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.195:40645 (local address 127.24.227.254:34525)
0923 23:14:58.868602 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:58.868604 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:58.868779 (+ 175us) server_negotiation.cc:407] Connection header received
0923 23:14:58.868826 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:58.868828 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:58.868843 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:58.868907 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:58.869195 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:58.869684 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:58.870346 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:58.870523 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:58.870585 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:58.872793 (+ 2208us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.227.195:40645: BlockingRecv error: recv got EOF from 127.24.227.195:40645 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Sep 23 23:14:59 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.193@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 23:14:59 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.193@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 23:14:59.644007 2162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:59.639224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:34525 (local address 127.24.227.193:37471)
0923 23:14:59.639355 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:59.639368 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:59.639428 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:59.639593 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:59.639594 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:59.639776 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:14:59.639819 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:59.639821 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:59.640576 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:59.640577 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:59.641115 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:59.641116 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:59.641501 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:59.641502 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:59.641558 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:59.641980 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:14:59.641984 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:14:59.643889 (+ 1905us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.227.254:34525: Server oryx/127.24.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":85,"threads_started":1}
W20260923 23:14:59.644064 2163 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 23:14:59.639268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.193:37471 (local address 127.24.227.254:34525)
0923 23:14:59.639372 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:59.639374 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:59.639438 (+ 64us) server_negotiation.cc:407] Connection header received
0923 23:14:59.639487 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:59.639488 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:59.639500 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:59.639583 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:59.639934 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:59.640491 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:59.641206 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:59.641426 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:59.641503 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:59.643991 (+ 2488us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.227.193:37471: BlockingRecv error: recv got EOF from 127.24.227.193:37471 (error 108)
Metrics: {"server-negotiator.queue_time_us":82,"thread_start_us":56,"threads_started":1}
Sep 23 23:14:59 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.194@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 23:14:59 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.194@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 23:14:59.765085 2164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:59.760367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:34525 (local address 127.24.227.194:43001)
0923 23:14:59.760590 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:59.760609 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:59.760664 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:59.760799 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:59.760800 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:59.760953 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:14:59.761000 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:59.761001 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:59.761881 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:59.761882 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:59.762459 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:59.762462 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:59.762853 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:59.762854 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:59.762893 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:59.763316 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:14:59.763320 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:14:59.765027 (+ 1707us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.227.254:34525: Server oryx/127.24.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":175,"threads_started":1}
W20260923 23:14:59.765179 2163 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 23:14:59.760439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.194:43001 (local address 127.24.227.254:34525)
0923 23:14:59.760510 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:59.760512 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:59.760664 (+ 152us) server_negotiation.cc:407] Connection header received
0923 23:14:59.760708 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:59.760709 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:59.760719 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:59.760776 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:59.761057 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:59.761820 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:59.762542 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:59.762788 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:59.762842 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:59.765114 (+ 2272us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.227.194:43001: BlockingRecv error: recv got EOF from 127.24.227.194:43001 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Sep 23 23:14:59 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.195@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 23:14:59 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.195@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 23:14:59.878050 2165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:14:59.873323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:34525 (local address 127.24.227.195:36267)
0923 23:14:59.873490 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0923 23:14:59.873500 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 23:14:59.873556 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:14:59.873688 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:14:59.873688 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:14:59.873850 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:14:59.873895 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:59.873897 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:59.874929 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:59.874930 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:59.875479 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:14:59.875481 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:14:59.875801 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:14:59.875802 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:14:59.875842 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:59.876322 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:14:59.876327 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:14:59.877994 (+ 1667us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.227.254:34525: Server oryx/127.24.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":106,"threads_started":1}
W20260923 23:14:59.878142 2163 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 23:14:59.873371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.195:36267 (local address 127.24.227.254:34525)
0923 23:14:59.873432 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0923 23:14:59.873434 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:14:59.873556 (+ 122us) server_negotiation.cc:407] Connection header received
0923 23:14:59.873597 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:14:59.873598 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:14:59.873609 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:14:59.873671 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:14:59.873955 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:59.874834 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:59.875556 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:14:59.875733 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:14:59.875786 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:14:59.878076 (+ 2290us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.227.195:36267: BlockingRecv error: recv got EOF from 127.24.227.195:36267 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Sep 23 23:15:00 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.193@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 23:15:00 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.193@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 23:15:00.648720 2166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:15:00.644496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:34525 (local address 127.24.227.193:39713)
0923 23:15:00.644609 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0923 23:15:00.644621 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 23:15:00.644689 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:15:00.644817 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:15:00.644818 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:15:00.644978 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:15:00.645026 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:15:00.645028 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:15:00.645672 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:15:00.645673 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:15:00.646189 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:15:00.646190 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:15:00.646502 (+ 312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:15:00.646503 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:15:00.646540 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:00.646970 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:15:00.646975 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:15:00.648663 (+ 1688us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.227.254:34525: Server oryx/127.24.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":52,"threads_started":1}
W20260923 23:15:00.648825 2167 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 23:15:00.644564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.193:39713 (local address 127.24.227.254:34525)
0923 23:15:00.644697 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0923 23:15:00.644699 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:15:00.644708 (+ 9us) server_negotiation.cc:407] Connection header received
0923 23:15:00.644730 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:15:00.644731 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:15:00.644741 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:15:00.644798 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:15:00.645092 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:00.645606 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:00.646257 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:00.646442 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:00.646489 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:00.648750 (+ 2261us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.227.193:39713: BlockingRecv error: recv got EOF from 127.24.227.193:39713 (error 108)
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":64,"threads_started":1}
Sep 23 23:15:00 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.194@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 23:15:00 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.194@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 23:15:00.770254 2168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:15:00.765635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:34525 (local address 127.24.227.194:56683)
0923 23:15:00.765760 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0923 23:15:00.765770 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 23:15:00.765819 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:15:00.765939 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:15:00.765940 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:15:00.766125 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:15:00.766175 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:15:00.766177 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:15:00.766938 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:15:00.766940 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:15:00.767536 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:15:00.767538 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:15:00.767859 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:15:00.767902 (+ 43us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:15:00.767979 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:00.768404 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:15:00.768409 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:15:00.770149 (+ 1740us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.227.254:34525: Server oryx/127.24.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":67,"threads_started":1}
W20260923 23:15:00.770290 2167 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 23:15:00.765728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.194:56683 (local address 127.24.227.254:34525)
0923 23:15:00.765795 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0923 23:15:00.765797 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:15:00.765813 (+ 16us) server_negotiation.cc:407] Connection header received
0923 23:15:00.765863 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:15:00.765863 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:15:00.765873 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:15:00.765926 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:15:00.766246 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:00.766847 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:00.767609 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:00.767791 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:00.767846 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:00.770234 (+ 2388us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.227.194:56683: BlockingRecv error: recv got EOF from 127.24.227.194:56683 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Sep 23 23:15:00 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.195@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 23:15:00 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.195@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 23:15:00.882826 2169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:15:00.878506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:34525 (local address 127.24.227.195:34527)
0923 23:15:00.878640 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0923 23:15:00.878649 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 23:15:00.878706 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:15:00.878829 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:15:00.878830 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:15:00.878979 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:15:00.879026 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:15:00.879028 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:15:00.879623 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:15:00.879624 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:15:00.880208 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:15:00.880211 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:15:00.880507 (+ 296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:15:00.880508 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:15:00.880547 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:00.880983 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:15:00.880987 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:15:00.882736 (+ 1749us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.227.254:34525: Server oryx/127.24.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":77,"threads_started":1}
W20260923 23:15:00.882862 2167 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 23:15:00.878571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.195:34527 (local address 127.24.227.254:34525)
0923 23:15:00.878613 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0923 23:15:00.878615 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:15:00.878710 (+ 95us) server_negotiation.cc:407] Connection header received
0923 23:15:00.878757 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:15:00.878757 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:15:00.878768 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:15:00.878808 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:15:00.879087 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:00.879558 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:00.880275 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:00.880448 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:00.880494 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:00.882805 (+ 2311us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.227.195:34527: BlockingRecv error: recv got EOF from 127.24.227.195:34527 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
Sep 23 23:15:01 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.193@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 23:15:01 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.193@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 23:15:01.659122 2170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:15:01.649169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:34525 (local address 127.24.227.193:44169)
0923 23:15:01.649299 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0923 23:15:01.649315 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0923 23:15:01.649360 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:15:01.649499 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:15:01.649500 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:15:01.649675 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:15:01.649736 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:15:01.649737 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:15:01.650368 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:15:01.650368 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:15:01.650891 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:15:01.650893 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:15:01.651219 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:15:01.651220 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:15:01.651290 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:01.657112 (+ 5822us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:15:01.657117 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:15:01.659054 (+ 1937us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.227.254:34525: Server oryx/127.24.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":77,"threads_started":1}
W20260923 23:15:01.659241 2171 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 23:15:01.649234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.193:44169 (local address 127.24.227.254:34525)
0923 23:15:01.649361 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0923 23:15:01.649363 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:15:01.649401 (+ 38us) server_negotiation.cc:407] Connection header received
0923 23:15:01.649421 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:15:01.649422 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:15:01.649431 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:15:01.649480 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:15:01.649800 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:01.650302 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:01.650958 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:01.651147 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:01.651195 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:01.659160 (+ 7965us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.227.193:44169: BlockingRecv error: recv got EOF from 127.24.227.193:44169 (error 108)
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":76,"threads_started":1}
Sep 23 23:15:01 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.194@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 23:15:01 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.194@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 23:15:01.775571 2172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:15:01.770756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:34525 (local address 127.24.227.194:47631)
0923 23:15:01.770893 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0923 23:15:01.770919 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0923 23:15:01.770981 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:15:01.771147 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:15:01.771148 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:15:01.771321 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:15:01.771380 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:15:01.771381 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:15:01.772067 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:15:01.772068 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:15:01.772609 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:15:01.772611 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:15:01.772961 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:15:01.772962 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:15:01.773052 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:01.773480 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:15:01.773485 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:15:01.775461 (+ 1976us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.227.254:34525: Server oryx/127.24.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":72,"threads_started":1}
W20260923 23:15:01.775630 2171 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 23:15:01.770870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.194:47631 (local address 127.24.227.254:34525)
0923 23:15:01.770953 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0923 23:15:01.770955 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:15:01.770963 (+ 8us) server_negotiation.cc:407] Connection header received
0923 23:15:01.771062 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:15:01.771063 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:15:01.771082 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:15:01.771140 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:15:01.771461 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:01.771992 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:01.772693 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:01.772890 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:01.772963 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:01.775560 (+ 2597us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.227.194:47631: BlockingRecv error: recv got EOF from 127.24.227.194:47631 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Sep 23 23:15:01 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.195@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 23:15:01 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.195@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 23:15:01.887678 2173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:15:01.883283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:34525 (local address 127.24.227.195:36031)
0923 23:15:01.883407 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0923 23:15:01.883418 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 23:15:01.883483 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:15:01.883613 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:15:01.883614 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:15:01.883773 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:15:01.883849 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:15:01.883851 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:15:01.884529 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:15:01.884530 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:15:01.885058 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:15:01.885060 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:15:01.885369 (+ 309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:15:01.885370 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:15:01.885408 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:01.885868 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:15:01.885873 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:15:01.887621 (+ 1748us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.227.254:34525: Server oryx/127.24.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":74,"threads_started":1}
W20260923 23:15:01.887742 2171 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 23:15:01.883365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.195:36031 (local address 127.24.227.254:34525)
0923 23:15:01.883409 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0923 23:15:01.883411 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:15:01.883469 (+ 58us) server_negotiation.cc:407] Connection header received
0923 23:15:01.883537 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:15:01.883538 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:15:01.883548 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:15:01.883591 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:15:01.883965 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:01.884464 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:01.885131 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:01.885309 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:01.885360 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:01.887696 (+ 2336us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.227.195:36031: BlockingRecv error: recv got EOF from 127.24.227.195:36031 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Sep 23 23:15:02 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.193@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 23:15:02 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.193@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 23:15:02.664244 2174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:15:02.659632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:34525 (local address 127.24.227.193:52597)
0923 23:15:02.659799 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0923 23:15:02.659813 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0923 23:15:02.659909 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:15:02.660046 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:15:02.660047 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:15:02.660204 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:15:02.660252 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:15:02.660254 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:15:02.660921 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:15:02.660921 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:15:02.661418 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:15:02.661419 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:15:02.661745 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:15:02.661746 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:15:02.661829 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:02.662289 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:15:02.662294 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:15:02.664157 (+ 1863us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.227.254:34525: Server oryx/127.24.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":71,"threads_started":1}
W20260923 23:15:02.664346 2175 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 23:15:02.659700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.193:52597 (local address 127.24.227.254:34525)
0923 23:15:02.659841 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0923 23:15:02.659843 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:15:02.659896 (+ 53us) server_negotiation.cc:407] Connection header received
0923 23:15:02.659957 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:15:02.659958 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:15:02.659968 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:15:02.660027 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:15:02.660354 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:02.660851 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:02.661492 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:02.661675 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:02.661731 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:02.664271 (+ 2540us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.227.193:52597: BlockingRecv error: recv got EOF from 127.24.227.193:52597 (error 108)
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":80,"threads_started":1}
Sep 23 23:15:02 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.194@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 23:15:02 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.194@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 23:15:02.780328 2176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:15:02.776062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:34525 (local address 127.24.227.194:47833)
0923 23:15:02.776196 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0923 23:15:02.776208 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 23:15:02.776287 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:15:02.776422 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:15:02.776423 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:15:02.776577 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:15:02.776623 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:15:02.776625 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:15:02.777264 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:15:02.777265 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:15:02.777771 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:15:02.777772 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:15:02.778081 (+ 309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:15:02.778082 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:15:02.778120 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:02.778583 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:15:02.778587 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:15:02.780270 (+ 1683us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.227.254:34525: Server oryx/127.24.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":83,"threads_started":1}
W20260923 23:15:02.780445 2175 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 23:15:02.776133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.194:47833 (local address 127.24.227.254:34525)
0923 23:15:02.776230 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0923 23:15:02.776232 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:15:02.776327 (+ 95us) server_negotiation.cc:407] Connection header received
0923 23:15:02.776350 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:15:02.776351 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:15:02.776361 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:15:02.776404 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:15:02.776684 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:02.777192 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:02.777845 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:02.778021 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:02.778068 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:02.780371 (+ 2303us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.227.194:47833: BlockingRecv error: recv got EOF from 127.24.227.194:47833 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Sep 23 23:15:02 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.195@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 23:15:02 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.195@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 23:15:02.892905 2177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:15:02.888243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:34525 (local address 127.24.227.195:53505)
0923 23:15:02.888392 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0923 23:15:02.888409 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0923 23:15:02.888486 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:15:02.888607 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:15:02.888608 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:15:02.888789 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:15:02.888834 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:15:02.888836 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:15:02.889504 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:15:02.889505 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:15:02.890038 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:15:02.890039 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:15:02.890407 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:15:02.890408 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:15:02.890483 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:02.890906 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:15:02.890912 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:15:02.892810 (+ 1898us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.227.254:34525: Server oryx/127.24.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":79,"threads_started":1}
W20260923 23:15:02.892954 2175 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 23:15:02.888297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.195:53505 (local address 127.24.227.254:34525)
0923 23:15:02.888360 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0923 23:15:02.888362 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:15:02.888469 (+ 107us) server_negotiation.cc:407] Connection header received
0923 23:15:02.888539 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:15:02.888540 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:15:02.888553 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:15:02.888604 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:15:02.888913 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:02.889423 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:02.890123 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:02.890321 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:02.890405 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:02.892891 (+ 2486us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.227.195:53505: BlockingRecv error: recv got EOF from 127.24.227.195:53505 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Sep 23 23:15:03 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.193@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 23:15:03 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.193@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 23:15:03.669644 2178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:15:03.664708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:34525 (local address 127.24.227.193:50459)
0923 23:15:03.664927 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0923 23:15:03.664938 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 23:15:03.665019 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:15:03.665170 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:15:03.665171 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:15:03.665410 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:15:03.665459 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:15:03.665461 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:15:03.666220 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:15:03.666221 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:15:03.666805 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:15:03.666807 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:15:03.667228 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:15:03.667229 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:15:03.667274 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:03.667754 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:15:03.667758 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:15:03.669588 (+ 1830us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.227.254:34525: Server oryx/127.24.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":83,"threads_started":1}
W20260923 23:15:03.669790 2179 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 23:15:03.664794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.193:50459 (local address 127.24.227.254:34525)
0923 23:15:03.664931 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0923 23:15:03.664934 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 23:15:03.665025 (+ 91us) server_negotiation.cc:407] Connection header received
0923 23:15:03.665074 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:15:03.665074 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:15:03.665086 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:15:03.665147 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:15:03.665528 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:03.666152 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:03.666909 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:03.667131 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:03.667198 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:03.669702 (+ 2504us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.227.193:50459: BlockingRecv error: recv got EOF from 127.24.227.193:50459 (error 108)
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":59,"threads_started":1}
Sep 23 23:15:03 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.194@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 23:15:03 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.194@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 23:15:03.785671 2180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:15:03.780836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:34525 (local address 127.24.227.194:50861)
0923 23:15:03.781041 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0923 23:15:03.781052 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 23:15:03.781143 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:15:03.781300 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:15:03.781301 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:15:03.781475 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:15:03.781519 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:15:03.781521 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:15:03.782351 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:15:03.782352 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:15:03.782863 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:15:03.782864 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:15:03.783295 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:15:03.783296 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:15:03.783370 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:03.783945 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:15:03.783951 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:15:03.785597 (+ 1646us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.227.254:34525: Server oryx/127.24.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":122,"threads_started":1}
W20260923 23:15:03.785761 2179 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 23:15:03.780932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.194:50861 (local address 127.24.227.254:34525)
0923 23:15:03.781026 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0923 23:15:03.781028 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:15:03.781124 (+ 96us) server_negotiation.cc:407] Connection header received
0923 23:15:03.781200 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:15:03.781201 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:15:03.781214 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:15:03.781283 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:15:03.781596 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:03.782260 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:03.782967 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:03.783186 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:03.783251 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:03.785699 (+ 2448us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.227.194:50861: BlockingRecv error: recv got EOF from 127.24.227.194:50861 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Sep 23 23:15:03 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.195@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 23:15:03 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.195@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 23:15:03.898116 2181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:15:03.893356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:34525 (local address 127.24.227.195:52427)
0923 23:15:03.893652 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0923 23:15:03.893667 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0923 23:15:03.893769 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:15:03.893940 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:15:03.893941 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:15:03.894131 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:15:03.894177 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:15:03.894179 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:15:03.894866 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:15:03.894867 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:15:03.895382 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:15:03.895383 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:15:03.895737 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:15:03.895737 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:15:03.895802 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:03.896344 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:15:03.896348 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:15:03.898022 (+ 1674us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.227.254:34525: Server oryx/127.24.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":198,"threads_started":1}
W20260923 23:15:03.898164 2179 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 23:15:03.893571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.195:52427 (local address 127.24.227.254:34525)
0923 23:15:03.893657 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0923 23:15:03.893659 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:15:03.893751 (+ 92us) server_negotiation.cc:407] Connection header received
0923 23:15:03.893823 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:15:03.893824 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:15:03.893838 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:15:03.893918 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:15:03.894258 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:03.894766 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:03.895464 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:03.895652 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:03.895736 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:03.898098 (+ 2362us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.227.195:52427: BlockingRecv error: recv got EOF from 127.24.227.195:52427 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Sep 23 23:15:04 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.193@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 23:15:04 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.193@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 23:15:04.674611 2182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:15:04.670083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:34525 (local address 127.24.227.193:33749)
0923 23:15:04.670217 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0923 23:15:04.670229 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 23:15:04.670283 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:15:04.670411 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:15:04.670411 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:15:04.670583 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:15:04.670631 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:15:04.670632 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:15:04.671253 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:15:04.671253 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:15:04.671738 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:15:04.671739 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:15:04.672096 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:15:04.672096 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:15:04.672138 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:04.672563 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:15:04.672567 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:15:04.674522 (+ 1955us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.227.254:34525: Server oryx/127.24.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":81,"threads_started":1}
W20260923 23:15:04.674679 2183 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 23:15:04.670167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.193:33749 (local address 127.24.227.254:34525)
0923 23:15:04.670295 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0923 23:15:04.670296 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 23:15:04.670303 (+ 7us) server_negotiation.cc:407] Connection header received
0923 23:15:04.670324 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:15:04.670324 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:15:04.670335 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:15:04.670395 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:15:04.670695 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:04.671187 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:04.671816 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:04.672035 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:04.672087 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:04.674612 (+ 2525us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.227.193:33749: BlockingRecv error: recv got EOF from 127.24.227.193:33749 (error 108)
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":68,"threads_started":1}
Sep 23 23:15:04 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.194@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 23:15:04 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.194@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 23:15:04.790649 2184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:15:04.786165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:34525 (local address 127.24.227.194:41101)
0923 23:15:04.786303 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0923 23:15:04.786321 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0923 23:15:04.786373 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:15:04.786492 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:15:04.786493 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:15:04.786659 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:15:04.786707 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:15:04.786709 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:15:04.787326 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:15:04.787326 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:15:04.787832 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:15:04.787833 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:15:04.788203 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:15:04.788204 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:15:04.788249 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:04.788699 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:15:04.788703 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:15:04.790552 (+ 1849us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.227.254:34525: Server oryx/127.24.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":93,"threads_started":1}
W20260923 23:15:04.790699 2183 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 23:15:04.786233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.194:41101 (local address 127.24.227.254:34525)
0923 23:15:04.786281 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0923 23:15:04.786283 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:15:04.786373 (+ 90us) server_negotiation.cc:407] Connection header received
0923 23:15:04.786417 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:15:04.786418 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:15:04.786428 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:15:04.786477 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:15:04.786764 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:04.787256 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:04.787950 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:04.788134 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:04.788187 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:04.790634 (+ 2447us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.227.194:41101: BlockingRecv error: recv got EOF from 127.24.227.194:41101 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
Sep 23 23:15:04 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.195@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 23:15:04 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.195@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 23:15:04.903155 2185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:15:04.898585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:34525 (local address 127.24.227.195:47425)
0923 23:15:04.898760 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0923 23:15:04.898790 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0923 23:15:04.898857 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:15:04.898988 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:15:04.898989 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:15:04.899154 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:15:04.899249 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:15:04.899251 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:15:04.899931 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:15:04.899932 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:15:04.900504 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:15:04.900506 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:15:04.900824 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:15:04.900825 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:15:04.900862 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:04.901321 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:15:04.901325 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:15:04.903089 (+ 1764us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.227.254:34525: Server oryx/127.24.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":95,"threads_started":1}
W20260923 23:15:04.903264 2183 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 23:15:04.898711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.195:47425 (local address 127.24.227.254:34525)
0923 23:15:04.898763 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0923 23:15:04.898765 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:15:04.898856 (+ 91us) server_negotiation.cc:407] Connection header received
0923 23:15:04.898910 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:15:04.898910 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:15:04.898923 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:15:04.898971 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:15:04.899330 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:04.899821 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:04.900580 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:04.900762 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:04.900814 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:04.903189 (+ 2375us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.227.195:47425: BlockingRecv error: recv got EOF from 127.24.227.195:47425 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
Sep 23 23:15:05 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.193@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 23:15:05 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.193@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 23:15:05.679889 2186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:15:05.675169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:34525 (local address 127.24.227.193:47265)
0923 23:15:05.675321 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0923 23:15:05.675341 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0923 23:15:05.675405 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:15:05.675571 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:15:05.675573 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:15:05.675734 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:15:05.675785 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:15:05.675786 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:15:05.676635 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:15:05.676636 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:15:05.677190 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:15:05.677192 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:15:05.677568 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:15:05.677568 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:15:05.677608 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:05.678083 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:15:05.678087 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:15:05.679811 (+ 1724us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.227.254:34525: Server oryx/127.24.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":69,"threads_started":1}
W20260923 23:15:05.680069 2187 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 23:15:05.675206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.193:47265 (local address 127.24.227.254:34525)
0923 23:15:05.675360 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0923 23:15:05.675362 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:15:05.675388 (+ 26us) server_negotiation.cc:407] Connection header received
0923 23:15:05.675456 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:15:05.675457 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:15:05.675468 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:15:05.675530 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:15:05.675950 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:05.676561 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:05.677291 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:05.677501 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:05.677560 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:05.680016 (+ 2456us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.227.193:47265: BlockingRecv error: recv got EOF from 127.24.227.193:47265 (error 108)
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":87,"threads_started":1}
Sep 23 23:15:05 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.194@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 23:15:05 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.194@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 23:15:05.796271 2188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:15:05.791246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:34525 (local address 127.24.227.194:39563)
0923 23:15:05.791442 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0923 23:15:05.791454 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 23:15:05.791522 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:15:05.791719 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:15:05.791720 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:15:05.792073 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:15:05.792123 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:15:05.792125 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:15:05.793000 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:15:05.793001 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:15:05.793584 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:15:05.793586 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:15:05.793919 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:15:05.793920 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:15:05.793957 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:05.794430 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:15:05.794434 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:15:05.796213 (+ 1779us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.227.254:34525: Server oryx/127.24.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":104,"threads_started":1}
W20260923 23:15:05.796397 2187 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 23:15:05.791378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.194:39563 (local address 127.24.227.254:34525)
0923 23:15:05.791457 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0923 23:15:05.791459 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:15:05.791508 (+ 49us) server_negotiation.cc:407] Connection header received
0923 23:15:05.791603 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:15:05.791604 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:15:05.791616 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:15:05.791686 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:15:05.792196 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:05.792917 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:05.793665 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:05.793850 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:05.793907 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:05.796292 (+ 2385us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.227.194:39563: BlockingRecv error: recv got EOF from 127.24.227.194:39563 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Sep 23 23:15:05 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.195@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 23:15:05 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.195@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 23:15:05.908207 2189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:15:05.903715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:34525 (local address 127.24.227.195:43089)
0923 23:15:05.903906 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0923 23:15:05.903920 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0923 23:15:05.903992 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:15:05.904100 (+ 108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:15:05.904101 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:15:05.904254 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:15:05.904302 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:15:05.904304 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:15:05.904903 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:15:05.904904 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:15:05.905474 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:15:05.905477 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:15:05.905796 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:15:05.905797 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:15:05.905834 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:05.906347 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:15:05.906353 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:15:05.908148 (+ 1795us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.227.254:34525: Server oryx/127.24.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":82,"threads_started":1}
W20260923 23:15:05.908387 2187 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 23:15:05.903890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.195:43089 (local address 127.24.227.254:34525)
0923 23:15:05.903941 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0923 23:15:05.903943 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:15:05.903975 (+ 32us) server_negotiation.cc:407] Connection header received
0923 23:15:05.904034 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:15:05.904035 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:15:05.904046 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:15:05.904089 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:15:05.904359 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:05.904835 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:05.905539 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:05.905734 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:05.905784 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:05.908314 (+ 2530us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.227.195:43089: BlockingRecv error: recv got EOF from 127.24.227.195:43089 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
Sep 23 23:15:06 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.193@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 23:15:06 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.193@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 23:15:06.691828 2190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:15:06.680593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:34525 (local address 127.24.227.193:32867)
0923 23:15:06.680764 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0923 23:15:06.680775 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 23:15:06.680826 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:15:06.680974 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:15:06.680975 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:15:06.681138 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:15:06.681186 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:15:06.681187 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:15:06.681962 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:15:06.681962 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:15:06.682640 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:15:06.682642 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:15:06.683053 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:15:06.683054 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:15:06.683090 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:06.690015 (+ 6925us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:15:06.690020 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:15:06.691761 (+ 1741us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.227.254:34525: Server oryx/127.24.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":105,"threads_started":1}
W20260923 23:15:06.691992 2191 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 23:15:06.680673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.193:32867 (local address 127.24.227.254:34525)
0923 23:15:06.680807 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0923 23:15:06.680809 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:15:06.680818 (+ 9us) server_negotiation.cc:407] Connection header received
0923 23:15:06.680889 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:15:06.680890 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:15:06.680900 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:15:06.680951 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:15:06.681250 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:06.681855 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:06.682713 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:06.682988 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:06.683043 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:06.691902 (+ 8859us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.227.193:32867: BlockingRecv error: recv got EOF from 127.24.227.193:32867 (error 108)
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":81,"threads_started":1}
Sep 23 23:15:06 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.194@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 23:15:06 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.194@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 23:15:06.801513 2192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:15:06.796686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:34525 (local address 127.24.227.194:41281)
0923 23:15:06.796849 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0923 23:15:06.796872 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0923 23:15:06.796949 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:15:06.797101 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:15:06.797102 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:15:06.797282 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:15:06.797325 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:15:06.797327 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:15:06.798017 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:15:06.798018 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:15:06.798546 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:15:06.798548 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:15:06.798973 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:15:06.798974 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:15:06.799035 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:06.799582 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:15:06.799588 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:15:06.801444 (+ 1856us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.227.254:34525: Server oryx/127.24.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":84,"threads_started":1}
W20260923 23:15:06.801636 2191 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 23:15:06.796763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.194:41281 (local address 127.24.227.254:34525)
0923 23:15:06.796831 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0923 23:15:06.796833 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:15:06.796930 (+ 97us) server_negotiation.cc:407] Connection header received
0923 23:15:06.797014 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:15:06.797015 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:15:06.797027 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:15:06.797095 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:15:06.797425 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:06.797937 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:06.798629 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:06.798890 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:06.798972 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:06.801533 (+ 2561us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.227.194:41281: BlockingRecv error: recv got EOF from 127.24.227.194:41281 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Sep 23 23:15:06 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.195@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 23:15:06 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.195@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 23:15:06.913450 2193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:15:06.908651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:34525 (local address 127.24.227.195:55339)
0923 23:15:06.908830 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0923 23:15:06.908841 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 23:15:06.908932 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:15:06.909064 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:15:06.909064 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:15:06.909249 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:15:06.909294 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:15:06.909296 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:15:06.909952 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:15:06.909953 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:15:06.910514 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:15:06.910515 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:15:06.910880 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:15:06.910881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:15:06.910938 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:06.911438 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:15:06.911443 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:15:06.913329 (+ 1886us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.227.254:34525: Server oryx/127.24.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":100,"threads_started":1}
W20260923 23:15:06.913496 2191 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 23:15:06.908731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.195:55339 (local address 127.24.227.254:34525)
0923 23:15:06.908807 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0923 23:15:06.908809 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:15:06.908913 (+ 104us) server_negotiation.cc:407] Connection header received
0923 23:15:06.908993 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:15:06.908994 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:15:06.909009 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:15:06.909059 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:15:06.909392 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:06.909866 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:06.910597 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:06.910788 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:06.910874 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:06.913448 (+ 2574us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.227.195:55339: BlockingRecv error: recv got EOF from 127.24.227.195:55339 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Sep 23 23:15:07 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.193@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 23:15:07 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.193@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 23:15:07.696738 2194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:15:07.692332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:34525 (local address 127.24.227.193:44831)
0923 23:15:07.692507 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0923 23:15:07.692518 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 23:15:07.692575 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:15:07.692691 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:15:07.692691 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:15:07.692852 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:15:07.692899 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:15:07.692900 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:15:07.693571 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:15:07.693572 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:15:07.694085 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:15:07.694087 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:15:07.694467 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:15:07.694468 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:15:07.694530 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:07.695033 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:15:07.695037 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:15:07.696683 (+ 1646us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.227.254:34525: Server oryx/127.24.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":127,"threads_started":1}
W20260923 23:15:07.696851 2195 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 23:15:07.692452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.193:44831 (local address 127.24.227.254:34525)
0923 23:15:07.692581 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0923 23:15:07.692583 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:15:07.692594 (+ 11us) server_negotiation.cc:407] Connection header received
0923 23:15:07.692617 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:15:07.692617 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:15:07.692628 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:15:07.692686 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:15:07.692963 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:07.693464 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:07.694170 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:07.694386 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:07.694466 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:07.696770 (+ 2304us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.227.193:44831: BlockingRecv error: recv got EOF from 127.24.227.193:44831 (error 108)
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":56,"threads_started":1}
Sep 23 23:15:07 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.194@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 23:15:07 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.194@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 23:15:07.806591 2196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:15:07.801952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:34525 (local address 127.24.227.194:46237)
0923 23:15:07.802156 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0923 23:15:07.802166 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 23:15:07.802243 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:15:07.802395 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:15:07.802396 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:15:07.802583 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:15:07.802626 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:15:07.802628 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:15:07.803269 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:15:07.803270 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:15:07.803807 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:15:07.803809 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:15:07.804234 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:15:07.804235 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:15:07.804292 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:07.804735 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:15:07.804739 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:15:07.806532 (+ 1793us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.227.254:34525: Server oryx/127.24.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":123,"threads_started":1}
W20260923 23:15:07.806715 2195 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 23:15:07.802040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.194:46237 (local address 127.24.227.254:34525)
0923 23:15:07.802111 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0923 23:15:07.802114 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 23:15:07.802230 (+ 116us) server_negotiation.cc:407] Connection header received
0923 23:15:07.802301 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:15:07.802302 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:15:07.802316 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:15:07.802387 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:15:07.802706 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:07.803184 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:07.803935 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:07.804130 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:07.804233 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:07.806637 (+ 2404us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.227.194:46237: BlockingRecv error: recv got EOF from 127.24.227.194:46237 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Sep 23 23:15:07 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.195@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 23:15:07 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.195@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 23:15:07.918712 2197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:15:07.913955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:34525 (local address 127.24.227.195:35223)
0923 23:15:07.914140 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0923 23:15:07.914154 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0923 23:15:07.914232 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:15:07.914385 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:15:07.914386 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:15:07.914564 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:15:07.914608 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:15:07.914609 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:15:07.915288 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:15:07.915289 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:15:07.915836 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:15:07.915837 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:15:07.916292 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:15:07.916294 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:15:07.916356 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:07.916841 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:15:07.916846 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:15:07.918641 (+ 1795us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.227.254:34525: Server oryx/127.24.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":98,"threads_started":1}
W20260923 23:15:07.918831 2195 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 23:15:07.914064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.195:35223 (local address 127.24.227.254:34525)
0923 23:15:07.914156 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0923 23:15:07.914158 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:15:07.914221 (+ 63us) server_negotiation.cc:407] Connection header received
0923 23:15:07.914294 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:15:07.914296 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:15:07.914309 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:15:07.914378 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:15:07.914687 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:07.915198 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:07.916001 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:07.916192 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:07.916281 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:07.918754 (+ 2473us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.227.195:35223: BlockingRecv error: recv got EOF from 127.24.227.195:35223 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Sep 23 23:15:08 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.193@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 23:15:08 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.193@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 23:15:08.702131 2198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:15:08.697237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:34525 (local address 127.24.227.193:38919)
0923 23:15:08.697425 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0923 23:15:08.697436 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 23:15:08.697497 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:15:08.697674 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:15:08.697675 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:15:08.697841 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:15:08.697888 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:15:08.697889 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:15:08.698709 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:15:08.698710 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:15:08.699291 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:15:08.699293 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:15:08.699653 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:15:08.699654 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:15:08.699718 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:08.700269 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:15:08.700274 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:15:08.702063 (+ 1789us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.227.254:34525: Server oryx/127.24.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":93,"threads_started":1}
W20260923 23:15:08.702208 2199 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 23:15:08.697327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.193:38919 (local address 127.24.227.254:34525)
0923 23:15:08.697456 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0923 23:15:08.697458 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:15:08.697481 (+ 23us) server_negotiation.cc:407] Connection header received
0923 23:15:08.697553 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:15:08.697553 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:15:08.697563 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:15:08.697655 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:15:08.697965 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:08.698647 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:08.699360 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:08.699563 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:08.699642 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:08.702155 (+ 2513us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.227.193:38919: BlockingRecv error: recv got EOF from 127.24.227.193:38919 (error 108)
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":65,"threads_started":1}
Sep 23 23:15:08 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.194@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 23:15:08 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.194@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 23:15:08.811687 2200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:15:08.807152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:34525 (local address 127.24.227.194:53489)
0923 23:15:08.807303 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0923 23:15:08.807316 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 23:15:08.807405 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:15:08.807535 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:15:08.807536 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:15:08.807715 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:15:08.807762 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:15:08.807764 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:15:08.808396 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:15:08.808397 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:15:08.808919 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:15:08.808921 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:15:08.809300 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:15:08.809300 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:15:08.809338 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:08.809857 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:15:08.809862 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:15:08.811630 (+ 1768us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.227.254:34525: Server oryx/127.24.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":65,"threads_started":1}
W20260923 23:15:08.811847 2199 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 23:15:08.807275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.194:53489 (local address 127.24.227.254:34525)
0923 23:15:08.807322 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0923 23:15:08.807324 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:15:08.807386 (+ 62us) server_negotiation.cc:407] Connection header received
0923 23:15:08.807474 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:15:08.807474 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:15:08.807486 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:15:08.807524 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:15:08.807825 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:08.808334 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:08.809021 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:08.809227 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:08.809292 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:08.811771 (+ 2479us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.227.194:53489: BlockingRecv error: recv got EOF from 127.24.227.194:53489 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Sep 23 23:15:08 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.195@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 23:15:08 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.195@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 23:15:08.923755 2201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:15:08.919194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:34525 (local address 127.24.227.195:59835)
0923 23:15:08.919370 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0923 23:15:08.919380 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 23:15:08.919457 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:15:08.919587 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:15:08.919588 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:15:08.919783 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:15:08.919830 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:15:08.919832 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:15:08.920499 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:15:08.920500 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:15:08.921039 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:15:08.921040 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:15:08.921367 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:15:08.921368 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:15:08.921409 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:08.921910 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:15:08.921916 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:15:08.923697 (+ 1781us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.227.254:34525: Server oryx/127.24.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":93,"threads_started":1}
W20260923 23:15:08.923900 2199 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 23:15:08.919265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.195:59835 (local address 127.24.227.254:34525)
0923 23:15:08.919358 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0923 23:15:08.919360 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:15:08.919467 (+ 107us) server_negotiation.cc:407] Connection header received
0923 23:15:08.919495 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:15:08.919496 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:15:08.919507 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:15:08.919555 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:15:08.919929 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:08.920430 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:08.921113 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:08.921297 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:08.921349 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:08.923784 (+ 2435us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.227.195:59835: BlockingRecv error: recv got EOF from 127.24.227.195:59835 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Sep 23 23:15:09 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.193@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 23:15:09 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.193@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 23:15:09.706990 2202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:15:09.702519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:34525 (local address 127.24.227.193:41031)
0923 23:15:09.702644 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0923 23:15:09.702656 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 23:15:09.702734 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:15:09.702865 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:15:09.702866 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:15:09.703025 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:15:09.703072 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:15:09.703074 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:15:09.703738 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:15:09.703739 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:15:09.704315 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:15:09.704317 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:15:09.704654 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:15:09.704655 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:15:09.704727 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:09.705179 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:15:09.705183 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:15:09.706921 (+ 1738us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.227.254:34525: Server oryx/127.24.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":51,"threads_started":1}
W20260923 23:15:09.707059 2203 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 23:15:09.702598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.193:41031 (local address 127.24.227.254:34525)
0923 23:15:09.702729 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0923 23:15:09.702731 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:15:09.702740 (+ 9us) server_negotiation.cc:407] Connection header received
0923 23:15:09.702777 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:15:09.702777 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:15:09.702787 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:15:09.702843 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:15:09.703136 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:09.703668 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:09.704382 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:09.704575 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:09.704654 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:09.707006 (+ 2352us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.227.193:41031: BlockingRecv error: recv got EOF from 127.24.227.193:41031 (error 108)
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":64,"threads_started":1}
Sep 23 23:15:09 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.194@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 23:15:09 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.194@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
W20260923 23:15:09.816771 2203 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 23:15:09.812235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.194:45667 (local address 127.24.227.254:34525)
0923 23:15:09.812282 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0923 23:15:09.812284 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:15:09.812340 (+ 56us) server_negotiation.cc:407] Connection header received
0923 23:15:09.812409 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:15:09.812410 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:15:09.812420 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:15:09.812460 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:15:09.812778 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:09.813257 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:09.813929 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:09.814110 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:09.814161 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:09.816718 (+ 2557us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.227.194:45667: BlockingRecv error: recv got EOF from 127.24.227.194:45667 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
I20260923 23:15:09.816767 2204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:15:09.812149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:34525 (local address 127.24.227.194:45667)
0923 23:15:09.812280 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0923 23:15:09.812297 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0923 23:15:09.812351 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:15:09.812486 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:15:09.812487 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:15:09.812649 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:15:09.812694 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:15:09.812696 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:15:09.813322 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:15:09.813323 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:15:09.813854 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:15:09.813855 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:15:09.814174 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:15:09.814175 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:15:09.814211 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:09.814823 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:15:09.814831 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:15:09.816632 (+ 1801us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.227.254:34525: Server oryx/127.24.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":92,"threads_started":1}
Sep 23 23:15:09 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.195@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 23:15:09 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.195@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 23:15:09.928839 2205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:15:09.924237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:34525 (local address 127.24.227.195:39291)
0923 23:15:09.924407 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0923 23:15:09.924420 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 23:15:09.924485 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:15:09.924699 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:15:09.924700 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:15:09.924870 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:15:09.924916 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:15:09.924918 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:15:09.925544 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:15:09.925545 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:15:09.926058 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:15:09.926060 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:15:09.926362 (+ 302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:15:09.926362 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:15:09.926400 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:09.926916 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:15:09.926927 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:15:09.928785 (+ 1858us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.227.254:34525: Server oryx/127.24.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":95,"threads_started":1}
W20260923 23:15:09.928966 2203 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 23:15:09.924326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.195:39291 (local address 127.24.227.254:34525)
0923 23:15:09.924538 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0923 23:15:09.924541 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 23:15:09.924551 (+ 10us) server_negotiation.cc:407] Connection header received
0923 23:15:09.924593 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:15:09.924594 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:15:09.924610 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:15:09.924673 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:15:09.924984 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:09.925468 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:09.926124 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:09.926302 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:09.926349 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:09.928889 (+ 2540us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.227.195:39291: BlockingRecv error: recv got EOF from 127.24.227.195:39291 (error 108)
Metrics: {"server-negotiator.queue_time_us":189}
Sep 23 23:15:10 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.193@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 23:15:10 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.193@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 23:15:10.711660 2206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:15:10.707482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:34525 (local address 127.24.227.193:38043)
0923 23:15:10.707610 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0923 23:15:10.707627 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0923 23:15:10.707675 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:15:10.707804 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:15:10.707805 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:15:10.707993 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:15:10.708040 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:15:10.708041 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:15:10.708644 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:15:10.708645 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:15:10.709155 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:15:10.709156 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:15:10.709449 (+ 293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:15:10.709450 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:15:10.709486 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:10.709921 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:15:10.709926 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:15:10.711605 (+ 1679us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.227.254:34525: Server oryx/127.24.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":76,"threads_started":1}
W20260923 23:15:10.711762 2207 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 23:15:10.707548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.193:38043 (local address 127.24.227.254:34525)
0923 23:15:10.707659 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0923 23:15:10.707661 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:15:10.707683 (+ 22us) server_negotiation.cc:407] Connection header received
0923 23:15:10.707724 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:15:10.707724 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:15:10.707734 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:15:10.707784 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:15:10.708098 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:10.708581 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:10.709215 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:10.709390 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:10.709436 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:10.711688 (+ 2252us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.227.193:38043: BlockingRecv error: recv got EOF from 127.24.227.193:38043 (error 108)
Metrics: {"server-negotiator.queue_time_us":90,"thread_start_us":61,"threads_started":1}
Sep 23 23:15:10 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.194@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
Sep 23 23:15:10 dist-test-slave-754n krb5kdc[1149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.227.194@KRBTEST.COM for oryx/127.24.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260923 23:15:10.821878 2208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:15:10.817282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:34525 (local address 127.24.227.194:52809)
0923 23:15:10.817423 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0923 23:15:10.817440 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0923 23:15:10.817494 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:15:10.817625 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:15:10.817626 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:15:10.817783 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:15:10.817831 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:15:10.817832 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:15:10.818537 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:15:10.818538 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:15:10.819050 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:15:10.819052 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:15:10.819517 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:15:10.819518 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:15:10.819599 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:10.820086 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:15:10.820091 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:15:10.821800 (+ 1709us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.227.254:34525: Server oryx/127.24.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":64,"threads_started":1}
W20260923 23:15:10.821943 2207 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 23:15:10.817364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.194:52809 (local address 127.24.227.254:34525)
0923 23:15:10.817429 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0923 23:15:10.817430 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 23:15:10.817500 (+ 70us) server_negotiation.cc:407] Connection header received
0923 23:15:10.817543 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:15:10.817543 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:15:10.817554 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:15:10.817608 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:15:10.817950 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:10.818448 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:10.819122 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:10.819305 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:10.819362 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:10.821899 (+ 2537us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.227.194:52809: BlockingRecv error: recv got EOF from 127.24.227.194:52809 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
I20260923 23:15:10.851550 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 1733
I20260923 23:15:10.855759 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 1884
I20260923 23:15:10.859948 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 2016
I20260923 23:15:10.863832 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 1662
2026-09-23T23:15:10Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17324 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-09-23T23:15:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T23:15:10Z Disabled control of system clock
I20260923 23:15:10.890910 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790205259943518-25487-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790205259943518-25487-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790205259943518-25487-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.254:41501
--webserver_interface=127.24.227.254
--webserver_port=0
--builtin_ntp_servers=127.24.227.212:33003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.227.254:41501
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260923 23:15:10.968431 2212 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:15:10.968611 2212 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:15:10.968631 2212 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:15:10.969696 2212 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 23:15:10.969730 2212 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:15:10.969743 2212 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:15:10.969755 2212 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 23:15:10.969767 2212 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 23:15:10.970979 2212 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:33003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790205259943518-25487-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790205259943518-25487-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.227.254:41501
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.254:41501
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.227.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2212
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790205259943518-25487-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:15:10.971153 2212 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:15:10.971369 2212 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:15:10.976078 2220 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:15:10.976061 2218 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:15:10.976047 2217 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:15:10.976114 2212 server_base.cc:1034] running on GCE node
I20260923 23:15:10.976449 2212 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:15:10.976671 2212 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:15:10.977802 2212 hybrid_clock.cc:648] HybridClock initialized: now 1790205310977790 us; error 27 us; skew 500 ppm
I20260923 23:15:10.979036 2212 webserver.cc:460] Webserver started at http://127.24.227.254:33391/ using document root <none> and password file <none>
I20260923 23:15:10.979264 2212 fs_manager.cc:359] Metadata directory not provided
I20260923 23:15:10.979310 2212 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:15:10.979455 2212 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:15:10.980963 2212 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790205259943518-25487-0/minicluster-data/master-0/data/instance:
uuid: "2b4657f154c746d69d59f73e32a5975a"
format_stamp: "Formatted at 2026-09-23 23:15:10 on dist-test-slave-754n"
server_key: "a19261d071604255b94deff68dc7c856"
server_key_iv: "2865d285b616f617515c81a89b0ed261"
server_key_version: "encryptionkey@0"
I20260923 23:15:10.981431 2212 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790205259943518-25487-0/minicluster-data/master-0/wal/instance:
uuid: "2b4657f154c746d69d59f73e32a5975a"
format_stamp: "Formatted at 2026-09-23 23:15:10 on dist-test-slave-754n"
server_key: "a19261d071604255b94deff68dc7c856"
server_key_iv: "2865d285b616f617515c81a89b0ed261"
server_key_version: "encryptionkey@0"
I20260923 23:15:10.982998 2212 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 23:15:10.983716 2226 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:15:10.983909 2212 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 23:15:10.983999 2212 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790205259943518-25487-0/minicluster-data/master-0/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790205259943518-25487-0/minicluster-data/master-0/wal
uuid: "2b4657f154c746d69d59f73e32a5975a"
format_stamp: "Formatted at 2026-09-23 23:15:10 on dist-test-slave-754n"
server_key: "a19261d071604255b94deff68dc7c856"
server_key_iv: "2865d285b616f617515c81a89b0ed261"
server_key_version: "encryptionkey@0"
I20260923 23:15:10.984071 2212 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790205259943518-25487-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790205259943518-25487-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790205259943518-25487-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:15:11.006218 2212 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:15:11.006491 2212 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:15:11.006651 2212 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:15:11.010636 2212 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.254:41501
I20260923 23:15:11.010672 2278 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.254:41501 every 8 connection(s)
I20260923 23:15:11.010998 2212 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
I20260923 23:15:11.011608 2279 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:15:11.014096 2279 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b4657f154c746d69d59f73e32a5975a: Bootstrap starting.
I20260923 23:15:11.014640 2279 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2b4657f154c746d69d59f73e32a5975a: Neither blocks nor log segments found. Creating new log.
I20260923 23:15:11.014743 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 2212
I20260923 23:15:11.014850 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790205259943518-25487-0/minicluster-data/master-0/wal/instance
I20260923 23:15:11.014982 2279 log.cc:824] T 00000000000000000000000000000000 P 2b4657f154c746d69d59f73e32a5975a: Log is configured to *not* fsync() on all Append() calls
I20260923 23:15:11.015056 25487 external_mini_cluster.cc:1428] Setting key 8bb84bfa5b4a687f9367c5dca7ede27c
I20260923 23:15:11.015793 2279 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b4657f154c746d69d59f73e32a5975a: No bootstrap required, opened a new log
I20260923 23:15:11.017520 2279 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2b4657f154c746d69d59f73e32a5975a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b4657f154c746d69d59f73e32a5975a" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41501 } }
I20260923 23:15:11.017702 2279 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2b4657f154c746d69d59f73e32a5975a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:15:11.017737 2279 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2b4657f154c746d69d59f73e32a5975a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b4657f154c746d69d59f73e32a5975a, State: Initialized, Role: FOLLOWER
I20260923 23:15:11.017897 2279 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2b4657f154c746d69d59f73e32a5975a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b4657f154c746d69d59f73e32a5975a" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41501 } }
I20260923 23:15:11.017997 2279 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2b4657f154c746d69d59f73e32a5975a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 23:15:11.018028 2279 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2b4657f154c746d69d59f73e32a5975a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 23:15:11.018074 2279 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2b4657f154c746d69d59f73e32a5975a [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:15:11.018996 2279 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2b4657f154c746d69d59f73e32a5975a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b4657f154c746d69d59f73e32a5975a" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41501 } }
I20260923 23:15:11.019131 2279 leader_election.cc:302] T 00000000000000000000000000000000 P 2b4657f154c746d69d59f73e32a5975a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2b4657f154c746d69d59f73e32a5975a; no voters:
I20260923 23:15:11.019317 2279 leader_election.cc:288] T 00000000000000000000000000000000 P 2b4657f154c746d69d59f73e32a5975a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 23:15:11.019351 2284 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2b4657f154c746d69d59f73e32a5975a [term 1 FOLLOWER]: Leader election won for term 1
I20260923 23:15:11.019577 2279 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2b4657f154c746d69d59f73e32a5975a [sys.catalog]: configured and running, proceeding with master startup.
I20260923 23:15:11.019858 2284 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2b4657f154c746d69d59f73e32a5975a [term 1 LEADER]: Becoming Leader. State: Replica: 2b4657f154c746d69d59f73e32a5975a, State: Running, Role: LEADER
I20260923 23:15:11.019979 2284 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2b4657f154c746d69d59f73e32a5975a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b4657f154c746d69d59f73e32a5975a" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41501 } }
I20260923 23:15:11.020345 2285 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b4657f154c746d69d59f73e32a5975a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2b4657f154c746d69d59f73e32a5975a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b4657f154c746d69d59f73e32a5975a" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41501 } } }
I20260923 23:15:11.020443 2285 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b4657f154c746d69d59f73e32a5975a [sys.catalog]: This master's current role is: LEADER
I20260923 23:15:11.020358 2286 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b4657f154c746d69d59f73e32a5975a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2b4657f154c746d69d59f73e32a5975a. Latest consensus state: current_term: 1 leader_uuid: "2b4657f154c746d69d59f73e32a5975a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b4657f154c746d69d59f73e32a5975a" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41501 } } }
I20260923 23:15:11.020545 2286 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b4657f154c746d69d59f73e32a5975a [sys.catalog]: This master's current role is: LEADER
I20260923 23:15:11.020740 2283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:15:11.015528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55114 (local address 127.24.227.254:41501)
0923 23:15:11.015742 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0923 23:15:11.015756 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0923 23:15:11.015782 (+ 26us) server_negotiation.cc:407] Connection header received
0923 23:15:11.016231 (+ 449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:15:11.016237 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:15:11.016292 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:15:11.016404 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:15:11.016643 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:11.017449 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:11.018645 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:11.019022 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:11.019153 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:11.019870 (+ 717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:15:11.019924 (+ 54us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:15:11.019924 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 23:15:11.020114 (+ 190us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:15:11.020150 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 23:15:11.020164 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:15:11.020229 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0923 23:15:11.020435 (+ 206us) server_negotiation.cc:299] Negotiation successful
0923 23:15:11.020513 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":117,"threads_started":1}
W20260923 23:15:11.021528 2299 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2b4657f154c746d69d59f73e32a5975a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 23:15:11.021585 2299 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 23:15:11.021656 2293 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 23:15:11.021826 2293 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 23:15:11.023195 2293 catalog_manager.cc:1347] Generated new cluster ID: 010dd8ceb56a4e36859e2496266c13b8
I20260923 23:15:11.023249 2293 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 23:15:11.028476 2293 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 23:15:11.029090 2293 catalog_manager.cc:1504] Loading token signing keys...
I20260923 23:15:11.038157 2293 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2b4657f154c746d69d59f73e32a5975a: Generated new TSK 0
I20260923 23:15:11.038347 2293 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 23:15:11.045091 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790205259943518-25487-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790205259943518-25487-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790205259943518-25487-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790205259943518-25487-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.227.193:0
--local_ip_for_outbound_sockets=127.24.227.193
--webserver_interface=127.24.227.193
--webserver_port=0
--tserver_master_addrs=127.24.227.254:41501
--builtin_ntp_servers=127.24.227.212:33003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260923 23:15:11.139945 2303 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:15:11.140146 2303 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:15:11.140178 2303 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:15:11.141413 2303 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:15:11.141475 2303 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:15:11.141516 2303 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.227.193
I20260923 23:15:11.142828 2303 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:33003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790205259943518-25487-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790205259943518-25487-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.227.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790205259943518-25487-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.227.193
--webserver_port=0
--tserver_master_addrs=127.24.227.254:41501
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2303
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.227.193
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790205259943518-25487-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:15:11.143077 2303 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:15:11.143345 2303 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:15:11.149143 2309 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:15:11.149185 2303 server_base.cc:1034] running on GCE node
W20260923 23:15:11.149150 2308 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:15:11.149302 2311 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:15:11.149489 2303 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:15:11.149765 2303 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:15:11.150957 2303 hybrid_clock.cc:648] HybridClock initialized: now 1790205311150951 us; error 36 us; skew 500 ppm
I20260923 23:15:11.152364 2303 webserver.cc:460] Webserver started at http://127.24.227.193:46615/ using document root <none> and password file <none>
I20260923 23:15:11.152603 2303 fs_manager.cc:359] Metadata directory not provided
I20260923 23:15:11.152666 2303 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:15:11.152793 2303 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:15:11.154167 2303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790205259943518-25487-0/minicluster-data/ts-0/data/instance:
uuid: "8d32425d3b6f46a28095c812bb0f80ac"
format_stamp: "Formatted at 2026-09-23 23:15:11 on dist-test-slave-754n"
server_key: "f257008c3e57862ffe7e05b2f70b0bb0"
server_key_iv: "b856b4047ad8b4f93180a4d7909e4328"
server_key_version: "encryptionkey@0"
I20260923 23:15:11.154567 2303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790205259943518-25487-0/minicluster-data/ts-0/wal/instance:
uuid: "8d32425d3b6f46a28095c812bb0f80ac"
format_stamp: "Formatted at 2026-09-23 23:15:11 on dist-test-slave-754n"
server_key: "f257008c3e57862ffe7e05b2f70b0bb0"
server_key_iv: "b856b4047ad8b4f93180a4d7909e4328"
server_key_version: "encryptionkey@0"
I20260923 23:15:11.156140 2303 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 23:15:11.156877 2317 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:15:11.157052 2303 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 23:15:11.157130 2303 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790205259943518-25487-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790205259943518-25487-0/minicluster-data/ts-0/wal
uuid: "8d32425d3b6f46a28095c812bb0f80ac"
format_stamp: "Formatted at 2026-09-23 23:15:11 on dist-test-slave-754n"
server_key: "f257008c3e57862ffe7e05b2f70b0bb0"
server_key_iv: "b856b4047ad8b4f93180a4d7909e4328"
server_key_version: "encryptionkey@0"
I20260923 23:15:11.157188 2303 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790205259943518-25487-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790205259943518-25487-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790205259943518-25487-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:15:11.177513 2303 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:15:11.177805 2303 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:15:11.177912 2303 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:15:11.178109 2303 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 23:15:11.178342 2303 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 23:15:11.178368 2303 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 23:15:11.178391 2303 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 23:15:11.178403 2303 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 23:15:11.183996 2303 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.193:38253
I20260923 23:15:11.184044 2430 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.193:38253 every 8 connection(s)
I20260923 23:15:11.184335 2303 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790205259943518-25487-0/minicluster-data/ts-0/data/info.pb
I20260923 23:15:11.189170 2283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:15:11.184996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.193:47451 (local address 127.24.227.254:41501)
0923 23:15:11.185043 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0923 23:15:11.185046 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 23:15:11.185263 (+ 217us) server_negotiation.cc:407] Connection header received
0923 23:15:11.185744 (+ 481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:15:11.185745 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:15:11.185753 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:15:11.185805 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:15:11.186481 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:11.186968 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:11.188001 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:11.188184 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:11.188233 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:11.188949 (+ 716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:15:11.188950 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:15:11.188951 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 23:15:11.188954 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:15:11.188966 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 23:15:11.188970 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:15:11.189013 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0923 23:15:11.189097 (+ 84us) server_negotiation.cc:299] Negotiation successful
0923 23:15:11.189116 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260923 23:15:11.189342 2433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:15:11.184988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41501 (local address 127.24.227.193:47451)
0923 23:15:11.185168 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0923 23:15:11.185200 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0923 23:15:11.185666 (+ 466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:15:11.185854 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:15:11.185857 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:15:11.185864 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:15:11.186392 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:15:11.186397 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:15:11.187041 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:15:11.187041 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:15:11.187929 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:15:11.187931 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:15:11.188248 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:15:11.188249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:15:11.188350 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:11.188813 (+ 463us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 23:15:11.188814 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:15:11.188839 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 23:15:11.188860 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:15:11.189024 (+ 164us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:15:11.189031 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:15:11.189034 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 23:15:11.189081 (+ 47us) client_negotiation.cc:240] Negotiation successful
0923 23:15:11.189152 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":95,"threads_started":1}
I20260923 23:15:11.189776 2431 heartbeater.cc:347] Connected to a master server at 127.24.227.254:41501
I20260923 23:15:11.189877 2431 heartbeater.cc:464] Registering TS with master...
I20260923 23:15:11.190100 2431 heartbeater.cc:511] Master 127.24.227.254:41501 requested a full tablet report, sending...
I20260923 23:15:11.190217 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 2303
I20260923 23:15:11.190320 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790205259943518-25487-0/minicluster-data/ts-0/wal/instance
I20260923 23:15:11.190507 25487 external_mini_cluster.cc:1428] Setting key d87d2aa6147dac05d4542f98dd21219a
I20260923 23:15:11.190663 2243 ts_manager.cc:194] Registered new tserver with Master: 8d32425d3b6f46a28095c812bb0f80ac (127.24.227.193:38253)
I20260923 23:15:11.191511 2243 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.227.193:47451
I20260923 23:15:11.191533 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790205259943518-25487-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790205259943518-25487-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790205259943518-25487-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790205259943518-25487-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.227.194:0
--local_ip_for_outbound_sockets=127.24.227.194
--webserver_interface=127.24.227.194
--webserver_port=0
--tserver_master_addrs=127.24.227.254:41501
--builtin_ntp_servers=127.24.227.212:33003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260923 23:15:11.268618 2434 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:15:11.268827 2434 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:15:11.268859 2434 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:15:11.269991 2434 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:15:11.270051 2434 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:15:11.270092 2434 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.227.194
I20260923 23:15:11.271389 2434 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:33003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790205259943518-25487-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790205259943518-25487-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.227.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790205259943518-25487-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.227.194
--webserver_port=0
--tserver_master_addrs=127.24.227.254:41501
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2434
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.227.194
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790205259943518-25487-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:15:11.271626 2434 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:15:11.271941 2434 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:15:11.276598 2439 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:15:11.276614 2434 server_base.cc:1034] running on GCE node
W20260923 23:15:11.276571 2442 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:15:11.276582 2440 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:15:11.276929 2434 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:15:11.277171 2434 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:15:11.278331 2434 hybrid_clock.cc:648] HybridClock initialized: now 1790205311278323 us; error 31 us; skew 500 ppm
I20260923 23:15:11.279511 2434 webserver.cc:460] Webserver started at http://127.24.227.194:39527/ using document root <none> and password file <none>
I20260923 23:15:11.279762 2434 fs_manager.cc:359] Metadata directory not provided
I20260923 23:15:11.279834 2434 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:15:11.280009 2434 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:15:11.281114 2434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790205259943518-25487-0/minicluster-data/ts-1/data/instance:
uuid: "3fd08b2696744e5f88e528fb809f9ff7"
format_stamp: "Formatted at 2026-09-23 23:15:11 on dist-test-slave-754n"
server_key: "b6160952fba26e196eee4a786db1bd1b"
server_key_iv: "4fa55f10808fdbd7c247da5462c303a4"
server_key_version: "encryptionkey@0"
I20260923 23:15:11.281473 2434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790205259943518-25487-0/minicluster-data/ts-1/wal/instance:
uuid: "3fd08b2696744e5f88e528fb809f9ff7"
format_stamp: "Formatted at 2026-09-23 23:15:11 on dist-test-slave-754n"
server_key: "b6160952fba26e196eee4a786db1bd1b"
server_key_iv: "4fa55f10808fdbd7c247da5462c303a4"
server_key_version: "encryptionkey@0"
I20260923 23:15:11.282653 2434 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 23:15:11.283375 2448 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:15:11.283581 2434 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 23:15:11.283667 2434 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790205259943518-25487-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790205259943518-25487-0/minicluster-data/ts-1/wal
uuid: "3fd08b2696744e5f88e528fb809f9ff7"
format_stamp: "Formatted at 2026-09-23 23:15:11 on dist-test-slave-754n"
server_key: "b6160952fba26e196eee4a786db1bd1b"
server_key_iv: "4fa55f10808fdbd7c247da5462c303a4"
server_key_version: "encryptionkey@0"
I20260923 23:15:11.283746 2434 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790205259943518-25487-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790205259943518-25487-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790205259943518-25487-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:15:11.295104 2434 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:15:11.295346 2434 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:15:11.295475 2434 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:15:11.295671 2434 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 23:15:11.296043 2434 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 23:15:11.296094 2434 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 23:15:11.296133 2434 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 23:15:11.296162 2434 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 23:15:11.301990 2434 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.194:35491
I20260923 23:15:11.302023 2561 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.194:35491 every 8 connection(s)
I20260923 23:15:11.302318 2434 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790205259943518-25487-0/minicluster-data/ts-1/data/info.pb
I20260923 23:15:11.305888 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 2434
I20260923 23:15:11.306005 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790205259943518-25487-0/minicluster-data/ts-1/wal/instance
I20260923 23:15:11.306182 25487 external_mini_cluster.cc:1428] Setting key 9c3c2378d188443344c46052479b9731
I20260923 23:15:11.307098 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790205259943518-25487-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790205259943518-25487-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790205259943518-25487-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790205259943518-25487-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.227.195:0
--local_ip_for_outbound_sockets=127.24.227.195
--webserver_interface=127.24.227.195
--webserver_port=0
--tserver_master_addrs=127.24.227.254:41501
--builtin_ntp_servers=127.24.227.212:33003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260923 23:15:11.307461 2283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:15:11.303002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.194:59715 (local address 127.24.227.254:41501)
0923 23:15:11.303045 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0923 23:15:11.303047 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:15:11.303236 (+ 189us) server_negotiation.cc:407] Connection header received
0923 23:15:11.303689 (+ 453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:15:11.303690 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:15:11.303700 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:15:11.303749 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:15:11.304421 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:11.304878 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:11.305985 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:11.306238 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:11.306303 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:11.307130 (+ 827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:15:11.307131 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:15:11.307131 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 23:15:11.307135 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:15:11.307165 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 23:15:11.307170 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:15:11.307233 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0923 23:15:11.307351 (+ 118us) server_negotiation.cc:299] Negotiation successful
0923 23:15:11.307377 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260923 23:15:11.307662 2564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:15:11.302987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41501 (local address 127.24.227.194:59715)
0923 23:15:11.303141 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0923 23:15:11.303170 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0923 23:15:11.303608 (+ 438us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:15:11.303794 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:15:11.303797 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:15:11.303805 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:15:11.304327 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:15:11.304331 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:15:11.304981 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:15:11.304982 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:15:11.305875 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:15:11.305878 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:15:11.306309 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:15:11.306310 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:15:11.306447 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:11.306977 (+ 530us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 23:15:11.306978 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:15:11.307014 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 23:15:11.307041 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:15:11.307266 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:15:11.307273 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:15:11.307276 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 23:15:11.307339 (+ 63us) client_negotiation.cc:240] Negotiation successful
0923 23:15:11.307437 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":76,"threads_started":1}
I20260923 23:15:11.308154 2562 heartbeater.cc:347] Connected to a master server at 127.24.227.254:41501
I20260923 23:15:11.308228 2562 heartbeater.cc:464] Registering TS with master...
I20260923 23:15:11.308415 2562 heartbeater.cc:511] Master 127.24.227.254:41501 requested a full tablet report, sending...
I20260923 23:15:11.308768 2243 ts_manager.cc:194] Registered new tserver with Master: 3fd08b2696744e5f88e528fb809f9ff7 (127.24.227.194:35491)
I20260923 23:15:11.309113 2243 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.227.194:59715
W20260923 23:15:11.385121 2565 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:15:11.385319 2565 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:15:11.385350 2565 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:15:11.386562 2565 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:15:11.386612 2565 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:15:11.386653 2565 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.227.195
I20260923 23:15:11.388111 2565 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:33003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790205259943518-25487-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790205259943518-25487-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.227.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790205259943518-25487-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.227.195
--webserver_port=0
--tserver_master_addrs=127.24.227.254:41501
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2565
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.227.195
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790205259943518-25487-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:15:11.388348 2565 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:15:11.388654 2565 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:15:11.393227 2570 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:15:11.393234 2571 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:15:11.393235 2573 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:15:11.393378 2565 server_base.cc:1034] running on GCE node
I20260923 23:15:11.393600 2565 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:15:11.393764 2565 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:15:11.394886 2565 hybrid_clock.cc:648] HybridClock initialized: now 1790205311394876 us; error 26 us; skew 500 ppm
I20260923 23:15:11.396032 2565 webserver.cc:460] Webserver started at http://127.24.227.195:35061/ using document root <none> and password file <none>
I20260923 23:15:11.396246 2565 fs_manager.cc:359] Metadata directory not provided
I20260923 23:15:11.396288 2565 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:15:11.396421 2565 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:15:11.397401 2565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790205259943518-25487-0/minicluster-data/ts-2/data/instance:
uuid: "c0d0caa182aa452f8e567640d4f0788a"
format_stamp: "Formatted at 2026-09-23 23:15:11 on dist-test-slave-754n"
server_key: "df43f1cfc56c50d8cf5c944483a6ebc0"
server_key_iv: "d89deedc9ef4c83ab7281d688ea3bc41"
server_key_version: "encryptionkey@0"
I20260923 23:15:11.397718 2565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790205259943518-25487-0/minicluster-data/ts-2/wal/instance:
uuid: "c0d0caa182aa452f8e567640d4f0788a"
format_stamp: "Formatted at 2026-09-23 23:15:11 on dist-test-slave-754n"
server_key: "df43f1cfc56c50d8cf5c944483a6ebc0"
server_key_iv: "d89deedc9ef4c83ab7281d688ea3bc41"
server_key_version: "encryptionkey@0"
I20260923 23:15:11.398865 2565 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 23:15:11.399477 2579 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:15:11.399622 2565 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 23:15:11.399729 2565 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790205259943518-25487-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790205259943518-25487-0/minicluster-data/ts-2/wal
uuid: "c0d0caa182aa452f8e567640d4f0788a"
format_stamp: "Formatted at 2026-09-23 23:15:11 on dist-test-slave-754n"
server_key: "df43f1cfc56c50d8cf5c944483a6ebc0"
server_key_iv: "d89deedc9ef4c83ab7281d688ea3bc41"
server_key_version: "encryptionkey@0"
I20260923 23:15:11.399780 2565 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790205259943518-25487-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790205259943518-25487-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790205259943518-25487-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:15:11.423609 2565 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:15:11.423818 2565 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:15:11.423979 2565 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:15:11.424139 2565 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 23:15:11.424377 2565 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 23:15:11.424407 2565 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 23:15:11.424429 2565 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 23:15:11.424443 2565 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 23:15:11.429581 2565 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.195:43625
I20260923 23:15:11.429651 2692 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.195:43625 every 8 connection(s)
I20260923 23:15:11.429934 2565 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790205259943518-25487-0/minicluster-data/ts-2/data/info.pb
I20260923 23:15:11.432076 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 2565
I20260923 23:15:11.432176 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790205259943518-25487-0/minicluster-data/ts-2/wal/instance
I20260923 23:15:11.432349 25487 external_mini_cluster.cc:1428] Setting key f569dbe5ef467af2e576be6ea98cc1ea
I20260923 23:15:11.434830 2283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:15:11.430562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.195:56917 (local address 127.24.227.254:41501)
0923 23:15:11.430599 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0923 23:15:11.430601 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:15:11.430859 (+ 258us) server_negotiation.cc:407] Connection header received
0923 23:15:11.431351 (+ 492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:15:11.431352 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:15:11.431364 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:15:11.431422 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:15:11.432115 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:11.432696 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:11.433608 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:11.433811 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:11.433878 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:11.434538 (+ 660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:15:11.434539 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:15:11.434539 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 23:15:11.434543 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:15:11.434557 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 23:15:11.434561 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:15:11.434631 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0923 23:15:11.434736 (+ 105us) server_negotiation.cc:299] Negotiation successful
0923 23:15:11.434760 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260923 23:15:11.435038 2695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:15:11.430557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41501 (local address 127.24.227.195:56917)
0923 23:15:11.430739 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0923 23:15:11.430767 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0923 23:15:11.431247 (+ 480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:15:11.431496 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:15:11.431499 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:15:11.431506 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:15:11.432023 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:15:11.432027 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:15:11.432789 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:15:11.432790 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:15:11.433524 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:15:11.433526 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:15:11.433884 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:15:11.433886 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:15:11.434002 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:11.434397 (+ 395us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 23:15:11.434397 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:15:11.434441 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 23:15:11.434459 (+ 18us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:15:11.434631 (+ 172us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:15:11.434636 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:15:11.434638 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 23:15:11.434739 (+ 101us) client_negotiation.cc:240] Negotiation successful
0923 23:15:11.434827 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":89,"threads_started":1}
I20260923 23:15:11.435460 2693 heartbeater.cc:347] Connected to a master server at 127.24.227.254:41501
I20260923 23:15:11.435531 2693 heartbeater.cc:464] Registering TS with master...
I20260923 23:15:11.435726 2693 heartbeater.cc:511] Master 127.24.227.254:41501 requested a full tablet report, sending...
I20260923 23:15:11.436100 2243 ts_manager.cc:194] Registered new tserver with Master: c0d0caa182aa452f8e567640d4f0788a (127.24.227.195:43625)
I20260923 23:15:11.436470 2243 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.227.195:56917
I20260923 23:15:11.443799 25487 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260923 23:15:11.445426 2283 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 23:15:11.445040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55118 (local address 127.24.227.254:41501)
0923 23:15:11.445069 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0923 23:15:11.445070 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 23:15:11.445192 (+ 122us) server_negotiation.cc:407] Connection header received
0923 23:15:11.445241 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:15:11.445241 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:15:11.445248 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:15:11.445292 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:15:11.445386 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55118: BlockingRecv error: recv got EOF from 127.0.0.1:55118 (error 108)
Metrics: {"server-negotiator.queue_time_us":14}
I20260923 23:15:11.445530 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 2303
I20260923 23:15:11.450033 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 2434
I20260923 23:15:11.454125 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 2565
I20260923 23:15:11.458073 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 2212
2026-09-23T23:15:11Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (593 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-09-23T23:15:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T23:15:11Z Disabled control of system clock
I20260923 23:15:11.485015 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790205259943518-25487-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790205259943518-25487-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790205259943518-25487-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.254:36893
--webserver_interface=127.24.227.254
--webserver_port=0
--builtin_ntp_servers=127.24.227.212:32865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.227.254:36893
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260923 23:15:11.566246 2704 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:15:11.566425 2704 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:15:11.566452 2704 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:15:11.567520 2704 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 23:15:11.567556 2704 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:15:11.567569 2704 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:15:11.567581 2704 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 23:15:11.567593 2704 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 23:15:11.568913 2704 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:32865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790205259943518-25487-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790205259943518-25487-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.227.254:36893
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.254:36893
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.227.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2704
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790205259943518-25487-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:15:11.569110 2704 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:15:11.569341 2704 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:15:11.573904 2712 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:15:11.573949 2709 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:15:11.573907 2710 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:15:11.574097 2704 server_base.cc:1034] running on GCE node
I20260923 23:15:11.574441 2704 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:15:11.574709 2704 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:15:11.575907 2704 hybrid_clock.cc:648] HybridClock initialized: now 1790205311575848 us; error 29 us; skew 500 ppm
I20260923 23:15:11.576992 2704 webserver.cc:460] Webserver started at http://127.24.227.254:41151/ using document root <none> and password file <none>
I20260923 23:15:11.577226 2704 fs_manager.cc:359] Metadata directory not provided
I20260923 23:15:11.577267 2704 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:15:11.577409 2704 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:15:11.578397 2704 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790205259943518-25487-0/minicluster-data/master-0/data/instance:
uuid: "61567b76b7e544e18080875463ed256a"
format_stamp: "Formatted at 2026-09-23 23:15:11 on dist-test-slave-754n"
server_key: "749f887c0067b1438942410cd4336a47"
server_key_iv: "f50f1c0c400ee14878f3ce7f4b9f65b4"
server_key_version: "encryptionkey@0"
I20260923 23:15:11.578745 2704 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790205259943518-25487-0/minicluster-data/master-0/wal/instance:
uuid: "61567b76b7e544e18080875463ed256a"
format_stamp: "Formatted at 2026-09-23 23:15:11 on dist-test-slave-754n"
server_key: "749f887c0067b1438942410cd4336a47"
server_key_iv: "f50f1c0c400ee14878f3ce7f4b9f65b4"
server_key_version: "encryptionkey@0"
I20260923 23:15:11.579996 2704 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 23:15:11.580672 2718 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:15:11.580825 2704 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 23:15:11.580919 2704 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790205259943518-25487-0/minicluster-data/master-0/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790205259943518-25487-0/minicluster-data/master-0/wal
uuid: "61567b76b7e544e18080875463ed256a"
format_stamp: "Formatted at 2026-09-23 23:15:11 on dist-test-slave-754n"
server_key: "749f887c0067b1438942410cd4336a47"
server_key_iv: "f50f1c0c400ee14878f3ce7f4b9f65b4"
server_key_version: "encryptionkey@0"
I20260923 23:15:11.580996 2704 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790205259943518-25487-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790205259943518-25487-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790205259943518-25487-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:15:11.584169 2704 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:15:11.584412 2704 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:15:11.584550 2704 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:15:11.588168 2704 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.254:36893
I20260923 23:15:11.588194 2770 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.254:36893 every 8 connection(s)
I20260923 23:15:11.588505 2704 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
I20260923 23:15:11.588881 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 2704
I20260923 23:15:11.588991 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790205259943518-25487-0/minicluster-data/master-0/wal/instance
I20260923 23:15:11.589166 25487 external_mini_cluster.cc:1428] Setting key 5eb5a2562a4d9b69a3686b26fe19406d
I20260923 23:15:11.589197 2771 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:15:11.591688 2774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:15:11.589782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41994 (local address 127.24.227.254:36893)
0923 23:15:11.589959 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0923 23:15:11.589966 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 23:15:11.589992 (+ 26us) server_negotiation.cc:407] Connection header received
0923 23:15:11.590496 (+ 504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:15:11.590501 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:15:11.590537 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:15:11.590666 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:15:11.591292 (+ 626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:15:11.591296 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:15:11.591296 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 23:15:11.591300 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:15:11.591343 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 23:15:11.591353 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:15:11.591423 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0923 23:15:11.591494 (+ 71us) server_negotiation.cc:299] Negotiation successful
0923 23:15:11.591548 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":81,"threads_started":1}
I20260923 23:15:11.592142 2771 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61567b76b7e544e18080875463ed256a: Bootstrap starting.
I20260923 23:15:11.592715 2771 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 61567b76b7e544e18080875463ed256a: Neither blocks nor log segments found. Creating new log.
I20260923 23:15:11.592937 2771 log.cc:824] T 00000000000000000000000000000000 P 61567b76b7e544e18080875463ed256a: Log is configured to *not* fsync() on all Append() calls
I20260923 23:15:11.593485 2771 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61567b76b7e544e18080875463ed256a: No bootstrap required, opened a new log
I20260923 23:15:11.594673 2771 raft_consensus.cc:348] T 00000000000000000000000000000000 P 61567b76b7e544e18080875463ed256a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61567b76b7e544e18080875463ed256a" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 36893 } }
I20260923 23:15:11.594789 2771 raft_consensus.cc:374] T 00000000000000000000000000000000 P 61567b76b7e544e18080875463ed256a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:15:11.594810 2771 raft_consensus.cc:729] T 00000000000000000000000000000000 P 61567b76b7e544e18080875463ed256a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61567b76b7e544e18080875463ed256a, State: Initialized, Role: FOLLOWER
I20260923 23:15:11.594918 2771 consensus_queue.cc:260] T 00000000000000000000000000000000 P 61567b76b7e544e18080875463ed256a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61567b76b7e544e18080875463ed256a" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 36893 } }
I20260923 23:15:11.595000 2771 raft_consensus.cc:388] T 00000000000000000000000000000000 P 61567b76b7e544e18080875463ed256a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 23:15:11.595021 2771 raft_consensus.cc:482] T 00000000000000000000000000000000 P 61567b76b7e544e18080875463ed256a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 23:15:11.595054 2771 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 61567b76b7e544e18080875463ed256a [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:15:11.595471 2771 raft_consensus.cc:504] T 00000000000000000000000000000000 P 61567b76b7e544e18080875463ed256a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61567b76b7e544e18080875463ed256a" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 36893 } }
I20260923 23:15:11.595548 2771 leader_election.cc:302] T 00000000000000000000000000000000 P 61567b76b7e544e18080875463ed256a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 61567b76b7e544e18080875463ed256a; no voters:
I20260923 23:15:11.595652 2771 leader_election.cc:288] T 00000000000000000000000000000000 P 61567b76b7e544e18080875463ed256a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 23:15:11.595865 2776 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 61567b76b7e544e18080875463ed256a [term 1 FOLLOWER]: Leader election won for term 1
I20260923 23:15:11.595917 2771 sys_catalog.cc:564] T 00000000000000000000000000000000 P 61567b76b7e544e18080875463ed256a [sys.catalog]: configured and running, proceeding with master startup.
I20260923 23:15:11.596029 2776 raft_consensus.cc:686] T 00000000000000000000000000000000 P 61567b76b7e544e18080875463ed256a [term 1 LEADER]: Becoming Leader. State: Replica: 61567b76b7e544e18080875463ed256a, State: Running, Role: LEADER
I20260923 23:15:11.596122 2776 consensus_queue.cc:237] T 00000000000000000000000000000000 P 61567b76b7e544e18080875463ed256a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61567b76b7e544e18080875463ed256a" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 36893 } }
I20260923 23:15:11.596427 2778 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61567b76b7e544e18080875463ed256a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 61567b76b7e544e18080875463ed256a. Latest consensus state: current_term: 1 leader_uuid: "61567b76b7e544e18080875463ed256a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61567b76b7e544e18080875463ed256a" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 36893 } } }
I20260923 23:15:11.596510 2778 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61567b76b7e544e18080875463ed256a [sys.catalog]: This master's current role is: LEADER
I20260923 23:15:11.596661 2777 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61567b76b7e544e18080875463ed256a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "61567b76b7e544e18080875463ed256a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61567b76b7e544e18080875463ed256a" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 36893 } } }
I20260923 23:15:11.596721 2777 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61567b76b7e544e18080875463ed256a [sys.catalog]: This master's current role is: LEADER
W20260923 23:15:11.597824 2792 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 61567b76b7e544e18080875463ed256a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 23:15:11.597877 2792 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 23:15:11.597929 2783 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 23:15:11.598062 2783 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 23:15:11.599350 2783 catalog_manager.cc:1347] Generated new cluster ID: 93c49eecfb1b4c98969d5676bc21b4a6
I20260923 23:15:11.599401 2783 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 23:15:11.630218 2783 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 23:15:11.630378 2783 catalog_manager.cc:1504] Loading token signing keys...
I20260923 23:15:11.632550 2783 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 61567b76b7e544e18080875463ed256a: Generated new TSK 0
I20260923 23:15:11.632699 2783 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 23:15:11.639248 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790205259943518-25487-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790205259943518-25487-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790205259943518-25487-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790205259943518-25487-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.227.193:0
--local_ip_for_outbound_sockets=127.24.227.193
--webserver_interface=127.24.227.193
--webserver_port=0
--tserver_master_addrs=127.24.227.254:36893
--builtin_ntp_servers=127.24.227.212:32865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260923 23:15:11.738659 2795 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:15:11.738845 2795 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:15:11.738865 2795 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:15:11.740005 2795 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:15:11.740057 2795 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:15:11.740104 2795 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.227.193
I20260923 23:15:11.741400 2795 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:32865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790205259943518-25487-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790205259943518-25487-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.227.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790205259943518-25487-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.227.193
--webserver_port=0
--tserver_master_addrs=127.24.227.254:36893
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2795
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.227.193
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790205259943518-25487-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:15:11.741622 2795 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:15:11.741876 2795 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:15:11.746562 2803 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:15:11.746678 2795 server_base.cc:1034] running on GCE node
W20260923 23:15:11.746562 2800 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:15:11.746587 2801 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:15:11.746933 2795 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:15:11.747189 2795 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:15:11.748317 2795 hybrid_clock.cc:648] HybridClock initialized: now 1790205311748298 us; error 36 us; skew 500 ppm
I20260923 23:15:11.758925 2795 webserver.cc:460] Webserver started at http://127.24.227.193:35963/ using document root <none> and password file <none>
I20260923 23:15:11.759179 2795 fs_manager.cc:359] Metadata directory not provided
I20260923 23:15:11.759224 2795 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:15:11.759380 2795 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:15:11.760499 2795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790205259943518-25487-0/minicluster-data/ts-0/data/instance:
uuid: "57b87c0ee331429bbbb8397a185722bc"
format_stamp: "Formatted at 2026-09-23 23:15:11 on dist-test-slave-754n"
server_key: "39530726a7e52b6df4aa5491ebaaa44f"
server_key_iv: "3dde27b4efce658f27c36e59ace344d1"
server_key_version: "encryptionkey@0"
I20260923 23:15:11.760836 2795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790205259943518-25487-0/minicluster-data/ts-0/wal/instance:
uuid: "57b87c0ee331429bbbb8397a185722bc"
format_stamp: "Formatted at 2026-09-23 23:15:11 on dist-test-slave-754n"
server_key: "39530726a7e52b6df4aa5491ebaaa44f"
server_key_iv: "3dde27b4efce658f27c36e59ace344d1"
server_key_version: "encryptionkey@0"
I20260923 23:15:11.762089 2795 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 23:15:11.762768 2809 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:15:11.762952 2795 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 23:15:11.763027 2795 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790205259943518-25487-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790205259943518-25487-0/minicluster-data/ts-0/wal
uuid: "57b87c0ee331429bbbb8397a185722bc"
format_stamp: "Formatted at 2026-09-23 23:15:11 on dist-test-slave-754n"
server_key: "39530726a7e52b6df4aa5491ebaaa44f"
server_key_iv: "3dde27b4efce658f27c36e59ace344d1"
server_key_version: "encryptionkey@0"
I20260923 23:15:11.763113 2795 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790205259943518-25487-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790205259943518-25487-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790205259943518-25487-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:15:11.766279 2795 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:15:11.766530 2795 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:15:11.766662 2795 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:15:11.766862 2795 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 23:15:11.767153 2795 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 23:15:11.767184 2795 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 23:15:11.767231 2795 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 23:15:11.767256 2795 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 23:15:11.772907 2795 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.193:37933
I20260923 23:15:11.772955 2922 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.193:37933 every 8 connection(s)
I20260923 23:15:11.773229 2795 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790205259943518-25487-0/minicluster-data/ts-0/data/info.pb
I20260923 23:15:11.774433 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 2795
I20260923 23:15:11.774544 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790205259943518-25487-0/minicluster-data/ts-0/wal/instance
I20260923 23:15:11.774735 25487 external_mini_cluster.cc:1428] Setting key 13792d0c8dcf0147de807ebbc1808e65
I20260923 23:15:11.775658 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790205259943518-25487-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790205259943518-25487-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790205259943518-25487-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790205259943518-25487-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.227.194:0
--local_ip_for_outbound_sockets=127.24.227.194
--webserver_interface=127.24.227.194
--webserver_port=0
--tserver_master_addrs=127.24.227.254:36893
--builtin_ntp_servers=127.24.227.212:32865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260923 23:15:11.776185 2774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:15:11.773870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.193:36987 (local address 127.24.227.254:36893)
0923 23:15:11.773911 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0923 23:15:11.773914 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 23:15:11.774133 (+ 219us) server_negotiation.cc:407] Connection header received
0923 23:15:11.774573 (+ 440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:15:11.774575 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:15:11.774582 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:15:11.774635 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:15:11.775910 (+ 1275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:15:11.775912 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:15:11.775912 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 23:15:11.775918 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:15:11.775935 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 23:15:11.775940 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:15:11.776001 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0923 23:15:11.776072 (+ 71us) server_negotiation.cc:299] Negotiation successful
0923 23:15:11.776087 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260923 23:15:11.776301 2925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:15:11.773841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:36893 (local address 127.24.227.193:36987)
0923 23:15:11.774014 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0923 23:15:11.774052 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0923 23:15:11.774460 (+ 408us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:15:11.774710 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:15:11.774714 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:15:11.774728 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:15:11.775696 (+ 968us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 23:15:11.775697 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:15:11.775723 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 23:15:11.775775 (+ 52us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:15:11.776017 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:15:11.776021 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:15:11.776023 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 23:15:11.776077 (+ 54us) client_negotiation.cc:240] Negotiation successful
0923 23:15:11.776122 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":82,"threads_started":1}
I20260923 23:15:11.776719 2923 heartbeater.cc:347] Connected to a master server at 127.24.227.254:36893
I20260923 23:15:11.776809 2923 heartbeater.cc:464] Registering TS with master...
I20260923 23:15:11.776948 2923 heartbeater.cc:511] Master 127.24.227.254:36893 requested a full tablet report, sending...
I20260923 23:15:11.777504 2735 ts_manager.cc:194] Registered new tserver with Master: 57b87c0ee331429bbbb8397a185722bc (127.24.227.193:37933)
W20260923 23:15:11.853554 2926 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:15:11.853741 2926 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:15:11.853773 2926 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:15:11.854772 2926 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:15:11.854825 2926 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:15:11.854862 2926 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.227.194
I20260923 23:15:11.856084 2926 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:32865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790205259943518-25487-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790205259943518-25487-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.227.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790205259943518-25487-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.227.194
--webserver_port=0
--tserver_master_addrs=127.24.227.254:36893
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2926
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.227.194
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790205259943518-25487-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:15:11.856304 2926 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:15:11.856547 2926 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:15:11.861123 2934 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:15:11.861086 2932 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:15:11.861105 2931 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:15:11.861142 2926 server_base.cc:1034] running on GCE node
I20260923 23:15:11.861522 2926 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:15:11.861745 2926 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:15:11.862902 2926 hybrid_clock.cc:648] HybridClock initialized: now 1790205311862897 us; error 40 us; skew 500 ppm
I20260923 23:15:11.864090 2926 webserver.cc:460] Webserver started at http://127.24.227.194:41393/ using document root <none> and password file <none>
I20260923 23:15:11.864323 2926 fs_manager.cc:359] Metadata directory not provided
I20260923 23:15:11.864383 2926 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:15:11.864518 2926 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:15:11.865515 2926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790205259943518-25487-0/minicluster-data/ts-1/data/instance:
uuid: "d3d17f31ba2e449794afde0eb0c39ea0"
format_stamp: "Formatted at 2026-09-23 23:15:11 on dist-test-slave-754n"
server_key: "a19e2ea384437c871863986c4507283e"
server_key_iv: "c3cf599d1b687cbf6472817b86f11b51"
server_key_version: "encryptionkey@0"
I20260923 23:15:11.865818 2926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790205259943518-25487-0/minicluster-data/ts-1/wal/instance:
uuid: "d3d17f31ba2e449794afde0eb0c39ea0"
format_stamp: "Formatted at 2026-09-23 23:15:11 on dist-test-slave-754n"
server_key: "a19e2ea384437c871863986c4507283e"
server_key_iv: "c3cf599d1b687cbf6472817b86f11b51"
server_key_version: "encryptionkey@0"
I20260923 23:15:11.867137 2926 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 23:15:11.867826 2940 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:15:11.867988 2926 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 23:15:11.868074 2926 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790205259943518-25487-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790205259943518-25487-0/minicluster-data/ts-1/wal
uuid: "d3d17f31ba2e449794afde0eb0c39ea0"
format_stamp: "Formatted at 2026-09-23 23:15:11 on dist-test-slave-754n"
server_key: "a19e2ea384437c871863986c4507283e"
server_key_iv: "c3cf599d1b687cbf6472817b86f11b51"
server_key_version: "encryptionkey@0"
I20260923 23:15:11.868144 2926 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790205259943518-25487-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790205259943518-25487-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790205259943518-25487-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:15:11.871417 2926 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:15:11.871654 2926 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:15:11.871788 2926 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:15:11.872001 2926 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 23:15:11.872275 2926 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 23:15:11.872323 2926 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 23:15:11.872360 2926 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 23:15:11.872387 2926 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 23:15:11.878116 2926 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.194:46811
I20260923 23:15:11.878203 3053 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.194:46811 every 8 connection(s)
I20260923 23:15:11.878463 2926 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790205259943518-25487-0/minicluster-data/ts-1/data/info.pb
I20260923 23:15:11.880548 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 2926
I20260923 23:15:11.880684 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790205259943518-25487-0/minicluster-data/ts-1/wal/instance
I20260923 23:15:11.880895 25487 external_mini_cluster.cc:1428] Setting key 8bb40489ae6956ad3249b2466f2d0214
I20260923 23:15:11.881237 2774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:15:11.879099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.194:39257 (local address 127.24.227.254:36893)
0923 23:15:11.879152 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0923 23:15:11.879154 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:15:11.879338 (+ 184us) server_negotiation.cc:407] Connection header received
0923 23:15:11.879723 (+ 385us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:15:11.879724 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:15:11.879728 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:15:11.879768 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:15:11.880966 (+ 1198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:15:11.880967 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:15:11.880968 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 23:15:11.880972 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:15:11.880991 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 23:15:11.880995 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:15:11.881050 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0923 23:15:11.881144 (+ 94us) server_negotiation.cc:299] Negotiation successful
0923 23:15:11.881157 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260923 23:15:11.881456 3056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:15:11.879077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:36893 (local address 127.24.227.194:39257)
0923 23:15:11.879227 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0923 23:15:11.879261 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0923 23:15:11.879635 (+ 374us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:15:11.879813 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:15:11.879816 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:15:11.879824 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:15:11.880789 (+ 965us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 23:15:11.880790 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:15:11.880828 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 23:15:11.880860 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:15:11.881050 (+ 190us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:15:11.881055 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:15:11.881058 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 23:15:11.881152 (+ 94us) client_negotiation.cc:240] Negotiation successful
0923 23:15:11.881209 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":71,"threads_started":1}
I20260923 23:15:11.881866 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790205259943518-25487-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790205259943518-25487-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790205259943518-25487-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790205259943518-25487-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.227.195:0
--local_ip_for_outbound_sockets=127.24.227.195
--webserver_interface=127.24.227.195
--webserver_port=0
--tserver_master_addrs=127.24.227.254:36893
--builtin_ntp_servers=127.24.227.212:32865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260923 23:15:11.881960 3054 heartbeater.cc:347] Connected to a master server at 127.24.227.254:36893
I20260923 23:15:11.882045 3054 heartbeater.cc:464] Registering TS with master...
I20260923 23:15:11.882170 3054 heartbeater.cc:511] Master 127.24.227.254:36893 requested a full tablet report, sending...
I20260923 23:15:11.882553 2735 ts_manager.cc:194] Registered new tserver with Master: d3d17f31ba2e449794afde0eb0c39ea0 (127.24.227.194:46811)
W20260923 23:15:11.957144 3057 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:15:11.957357 3057 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:15:11.957391 3057 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:15:11.958479 3057 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:15:11.958536 3057 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:15:11.958576 3057 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.227.195
I20260923 23:15:11.959844 3057 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:32865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790205259943518-25487-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790205259943518-25487-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.227.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790205259943518-25487-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.227.195
--webserver_port=0
--tserver_master_addrs=127.24.227.254:36893
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3057
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.227.195
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790205259943518-25487-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:15:11.960100 3057 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:15:11.960351 3057 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:15:11.964905 3062 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:15:11.964910 3065 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:15:11.964919 3063 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:15:11.965240 3057 server_base.cc:1034] running on GCE node
I20260923 23:15:11.965457 3057 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:15:11.965696 3057 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:15:11.966838 3057 hybrid_clock.cc:648] HybridClock initialized: now 1790205311966823 us; error 31 us; skew 500 ppm
I20260923 23:15:11.968047 3057 webserver.cc:460] Webserver started at http://127.24.227.195:43873/ using document root <none> and password file <none>
I20260923 23:15:11.968256 3057 fs_manager.cc:359] Metadata directory not provided
I20260923 23:15:11.968313 3057 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:15:11.968456 3057 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:15:11.969408 3057 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790205259943518-25487-0/minicluster-data/ts-2/data/instance:
uuid: "00aa82ffb678440183bc114d72387b0b"
format_stamp: "Formatted at 2026-09-23 23:15:11 on dist-test-slave-754n"
server_key: "e717c54e957fe1337feffbe45d94383a"
server_key_iv: "dc215cb0474d6eb6e50aa2f547394052"
server_key_version: "encryptionkey@0"
I20260923 23:15:11.969738 3057 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790205259943518-25487-0/minicluster-data/ts-2/wal/instance:
uuid: "00aa82ffb678440183bc114d72387b0b"
format_stamp: "Formatted at 2026-09-23 23:15:11 on dist-test-slave-754n"
server_key: "e717c54e957fe1337feffbe45d94383a"
server_key_iv: "dc215cb0474d6eb6e50aa2f547394052"
server_key_version: "encryptionkey@0"
I20260923 23:15:11.971040 3057 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 23:15:11.971764 3071 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:15:11.971944 3057 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 23:15:11.972038 3057 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790205259943518-25487-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790205259943518-25487-0/minicluster-data/ts-2/wal
uuid: "00aa82ffb678440183bc114d72387b0b"
format_stamp: "Formatted at 2026-09-23 23:15:11 on dist-test-slave-754n"
server_key: "e717c54e957fe1337feffbe45d94383a"
server_key_iv: "dc215cb0474d6eb6e50aa2f547394052"
server_key_version: "encryptionkey@0"
I20260923 23:15:11.972119 3057 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790205259943518-25487-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790205259943518-25487-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790205259943518-25487-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:15:11.975155 3057 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:15:11.975379 3057 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:15:11.975504 3057 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:15:11.975714 3057 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 23:15:11.976042 3057 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 23:15:11.976089 3057 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 23:15:11.976127 3057 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 23:15:11.976154 3057 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 23:15:11.981583 3057 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.195:37293
I20260923 23:15:11.981624 3184 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.195:37293 every 8 connection(s)
I20260923 23:15:11.981887 3057 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790205259943518-25487-0/minicluster-data/ts-2/data/info.pb
I20260923 23:15:11.985018 2774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:15:11.982486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.195:41163 (local address 127.24.227.254:36893)
0923 23:15:11.982569 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0923 23:15:11.982571 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:15:11.982744 (+ 173us) server_negotiation.cc:407] Connection header received
0923 23:15:11.983351 (+ 607us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:15:11.983353 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:15:11.983360 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:15:11.983407 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:15:11.984759 (+ 1352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:15:11.984761 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:15:11.984761 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 23:15:11.984766 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:15:11.984783 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 23:15:11.984789 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:15:11.984849 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0923 23:15:11.984929 (+ 80us) server_negotiation.cc:299] Negotiation successful
0923 23:15:11.984944 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260923 23:15:11.985181 3187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:15:11.982450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:36893 (local address 127.24.227.195:41163)
0923 23:15:11.982634 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0923 23:15:11.982660 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0923 23:15:11.983217 (+ 557us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:15:11.983484 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:15:11.983488 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:15:11.983498 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:15:11.984604 (+ 1106us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 23:15:11.984605 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:15:11.984639 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 23:15:11.984672 (+ 33us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:15:11.984858 (+ 186us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:15:11.984862 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:15:11.984865 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 23:15:11.984937 (+ 72us) client_negotiation.cc:240] Negotiation successful
0923 23:15:11.984993 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":97,"threads_started":1}
I20260923 23:15:11.985771 3185 heartbeater.cc:347] Connected to a master server at 127.24.227.254:36893
I20260923 23:15:11.985894 3185 heartbeater.cc:464] Registering TS with master...
I20260923 23:15:11.986028 3185 heartbeater.cc:511] Master 127.24.227.254:36893 requested a full tablet report, sending...
I20260923 23:15:11.986346 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 3057
I20260923 23:15:11.986379 2735 ts_manager.cc:194] Registered new tserver with Master: 00aa82ffb678440183bc114d72387b0b (127.24.227.195:37293)
I20260923 23:15:11.986440 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790205259943518-25487-0/minicluster-data/ts-2/wal/instance
I20260923 23:15:11.986593 25487 external_mini_cluster.cc:1428] Setting key cd3def64bf55cb1955c5d1ce77be1210
I20260923 23:15:11.987360 25487 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260923 23:15:11.988969 2774 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 23:15:11.988530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42004 (local address 127.24.227.254:36893)
0923 23:15:11.988579 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0923 23:15:11.988581 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:15:11.988700 (+ 119us) server_negotiation.cc:407] Connection header received
0923 23:15:11.988747 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:15:11.988747 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:15:11.988751 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:15:11.988809 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:15:11.988908 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42004: BlockingRecv error: recv got EOF from 127.0.0.1:42004 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
I20260923 23:15:11.989063 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 2795
I20260923 23:15:11.994621 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 2926
I20260923 23:15:11.998473 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 3057
I20260923 23:15:12.002347 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 2704
2026-09-23T23:15:12Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (543 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790205259943518-25487-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 23:15:12 dist-test-slave-754n krb5kdc[3195](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 23:15:12 dist-test-slave-754n krb5kdc[3195](info): set up 2 sockets
Sep 23 23:15:12 dist-test-slave-754n krb5kdc[3195](info): commencing operation
krb5kdc: starting...
W20260923 23:15:13.024724 25487 mini_kdc.cc:121] Time spent starting KDC: real 1.004s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790205259943518-25487-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790205259943518-25487-0/krb5kdc/test-user.keytab.
Sep 23 23:15:13 dist-test-slave-754n krb5kdc[3195](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205313, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T23:15:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T23:15:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.227.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.227.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.227.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab.
Entry for principal kudu/127.24.227.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab.
I20260923 23:15:13.132850 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790205259943518-25487-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790205259943518-25487-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790205259943518-25487-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.254:43411
--webserver_interface=127.24.227.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab
--principal=kudu/127.24.227.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:37883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.227.254:43411
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:15:13.210541 3208 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:15:13.210721 3208 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:15:13.210741 3208 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:15:13.211822 3208 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 23:15:13.211856 3208 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:15:13.211897 3208 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:15:13.211910 3208 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 23:15:13.211925 3208 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 23:15:13.213279 3208 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:37883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790205259943518-25487-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790205259943518-25487-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.227.254:43411
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab
--principal=kudu/127.24.227.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.254:43411
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.227.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3208
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790205259943518-25487-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:15:13.213446 3208 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:15:13.213589 3208 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:15:13.217895 3214 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:15:13.217911 3213 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:15:13.217896 3216 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:15:13.217935 3208 server_base.cc:1034] running on GCE node
I20260923 23:15:13.218303 3208 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:15:13.218559 3208 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:15:13.219723 3208 hybrid_clock.cc:648] HybridClock initialized: now 1790205313219710 us; error 25 us; skew 500 ppm
Sep 23 23:15:13 dist-test-slave-754n krb5kdc[3195](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205313, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:15:13.222303 3208 init.cc:377] Logged in from keytab as kudu/127.24.227.254@KRBTEST.COM (short username kudu)
I20260923 23:15:13.222687 3208 webserver.cc:460] Webserver started at http://127.24.227.254:45181/ using document root <none> and password file <none>
I20260923 23:15:13.222918 3208 fs_manager.cc:359] Metadata directory not provided
I20260923 23:15:13.222978 3208 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:15:13.223121 3208 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:15:13.224114 3208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790205259943518-25487-0/minicluster-data/master-0/data/instance:
uuid: "05de63bdcc3b4d1cb45d1f00f7427e2c"
format_stamp: "Formatted at 2026-09-23 23:15:13 on dist-test-slave-754n"
server_key: "d3237c680efe8b71f7f6d11e5008b3b4"
server_key_iv: "1f0520c97844d1e68430210e7432a745"
server_key_version: "encryptionkey@0"
I20260923 23:15:13.224436 3208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790205259943518-25487-0/minicluster-data/master-0/wal/instance:
uuid: "05de63bdcc3b4d1cb45d1f00f7427e2c"
format_stamp: "Formatted at 2026-09-23 23:15:13 on dist-test-slave-754n"
server_key: "d3237c680efe8b71f7f6d11e5008b3b4"
server_key_iv: "1f0520c97844d1e68430210e7432a745"
server_key_version: "encryptionkey@0"
I20260923 23:15:13.225617 3208 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 23:15:13.226281 3223 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:15:13.226425 3208 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 23:15:13.226504 3208 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790205259943518-25487-0/minicluster-data/master-0/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790205259943518-25487-0/minicluster-data/master-0/wal
uuid: "05de63bdcc3b4d1cb45d1f00f7427e2c"
format_stamp: "Formatted at 2026-09-23 23:15:13 on dist-test-slave-754n"
server_key: "d3237c680efe8b71f7f6d11e5008b3b4"
server_key_iv: "1f0520c97844d1e68430210e7432a745"
server_key_version: "encryptionkey@0"
I20260923 23:15:13.226553 3208 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790205259943518-25487-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790205259943518-25487-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790205259943518-25487-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:15:13.247725 3208 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:15:13.249516 3208 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:15:13.249621 3208 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:15:13.253190 3208 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.254:43411
I20260923 23:15:13.253264 3275 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.254:43411 every 8 connection(s)
I20260923 23:15:13.253587 3208 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
I20260923 23:15:13.254855 3276 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:15:13.256788 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 3208
I20260923 23:15:13.256901 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790205259943518-25487-0/minicluster-data/master-0/wal/instance
I20260923 23:15:13.257045 3276 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05de63bdcc3b4d1cb45d1f00f7427e2c: Bootstrap starting.
I20260923 23:15:13.257081 25487 external_mini_cluster.cc:1428] Setting key f909564224d4a15bdddcfb347a22999e
I20260923 23:15:13.257675 3276 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 05de63bdcc3b4d1cb45d1f00f7427e2c: Neither blocks nor log segments found. Creating new log.
I20260923 23:15:13.257987 3276 log.cc:824] T 00000000000000000000000000000000 P 05de63bdcc3b4d1cb45d1f00f7427e2c: Log is configured to *not* fsync() on all Append() calls
I20260923 23:15:13.258636 3276 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05de63bdcc3b4d1cb45d1f00f7427e2c: No bootstrap required, opened a new log
I20260923 23:15:13.260371 3276 raft_consensus.cc:348] T 00000000000000000000000000000000 P 05de63bdcc3b4d1cb45d1f00f7427e2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05de63bdcc3b4d1cb45d1f00f7427e2c" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 43411 } }
I20260923 23:15:13.260490 3276 raft_consensus.cc:374] T 00000000000000000000000000000000 P 05de63bdcc3b4d1cb45d1f00f7427e2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:15:13.260512 3276 raft_consensus.cc:729] T 00000000000000000000000000000000 P 05de63bdcc3b4d1cb45d1f00f7427e2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05de63bdcc3b4d1cb45d1f00f7427e2c, State: Initialized, Role: FOLLOWER
I20260923 23:15:13.260637 3276 consensus_queue.cc:260] T 00000000000000000000000000000000 P 05de63bdcc3b4d1cb45d1f00f7427e2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05de63bdcc3b4d1cb45d1f00f7427e2c" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 43411 } }
I20260923 23:15:13.260732 3276 raft_consensus.cc:388] T 00000000000000000000000000000000 P 05de63bdcc3b4d1cb45d1f00f7427e2c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 23:15:13.260775 3276 raft_consensus.cc:482] T 00000000000000000000000000000000 P 05de63bdcc3b4d1cb45d1f00f7427e2c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 23:15:13.260844 3276 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 05de63bdcc3b4d1cb45d1f00f7427e2c [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:15:13.261368 3276 raft_consensus.cc:504] T 00000000000000000000000000000000 P 05de63bdcc3b4d1cb45d1f00f7427e2c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05de63bdcc3b4d1cb45d1f00f7427e2c" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 43411 } }
I20260923 23:15:13.261477 3276 leader_election.cc:302] T 00000000000000000000000000000000 P 05de63bdcc3b4d1cb45d1f00f7427e2c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 05de63bdcc3b4d1cb45d1f00f7427e2c; no voters:
I20260923 23:15:13.261668 3276 leader_election.cc:288] T 00000000000000000000000000000000 P 05de63bdcc3b4d1cb45d1f00f7427e2c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 23:15:13.261699 3281 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 05de63bdcc3b4d1cb45d1f00f7427e2c [term 1 FOLLOWER]: Leader election won for term 1
I20260923 23:15:13.261907 3276 sys_catalog.cc:564] T 00000000000000000000000000000000 P 05de63bdcc3b4d1cb45d1f00f7427e2c [sys.catalog]: configured and running, proceeding with master startup.
I20260923 23:15:13.262782 3281 raft_consensus.cc:686] T 00000000000000000000000000000000 P 05de63bdcc3b4d1cb45d1f00f7427e2c [term 1 LEADER]: Becoming Leader. State: Replica: 05de63bdcc3b4d1cb45d1f00f7427e2c, State: Running, Role: LEADER
I20260923 23:15:13.262881 3281 consensus_queue.cc:237] T 00000000000000000000000000000000 P 05de63bdcc3b4d1cb45d1f00f7427e2c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05de63bdcc3b4d1cb45d1f00f7427e2c" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 43411 } }
I20260923 23:15:13.263208 3281 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05de63bdcc3b4d1cb45d1f00f7427e2c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 05de63bdcc3b4d1cb45d1f00f7427e2c. Latest consensus state: current_term: 1 leader_uuid: "05de63bdcc3b4d1cb45d1f00f7427e2c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05de63bdcc3b4d1cb45d1f00f7427e2c" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 43411 } } }
I20260923 23:15:13.263267 3281 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05de63bdcc3b4d1cb45d1f00f7427e2c [sys.catalog]: This master's current role is: LEADER
Sep 23 23:15:13 dist-test-slave-754n krb5kdc[3195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205313, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
I20260923 23:15:13.263316 3282 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05de63bdcc3b4d1cb45d1f00f7427e2c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "05de63bdcc3b4d1cb45d1f00f7427e2c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05de63bdcc3b4d1cb45d1f00f7427e2c" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 43411 } } }
I20260923 23:15:13.263402 3282 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05de63bdcc3b4d1cb45d1f00f7427e2c [sys.catalog]: This master's current role is: LEADER
I20260923 23:15:13.264235 3288 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 23:15:13.264756 3288 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 23:15:13.266244 3288 catalog_manager.cc:1347] Generated new cluster ID: 86243a07f8ec4b3bbb3c7eac252ecfc1
I20260923 23:15:13.266294 3288 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 23:15:13.267719 3279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:15:13.257553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45992 (local address 127.24.227.254:43411)
0923 23:15:13.257721 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0923 23:15:13.257730 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 23:15:13.257762 (+ 32us) server_negotiation.cc:407] Connection header received
0923 23:15:13.258262 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:15:13.258269 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:15:13.258314 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:15:13.258457 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:15:13.259101 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:13.259945 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:13.260712 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:13.261232 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:13.261355 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:13.263831 (+ 2476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:15:13.263835 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:15:13.263835 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:15:13.263849 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:15:13.266099 (+ 2250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:15:13.266501 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:15:13.266503 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:15:13.266503 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:15:13.266590 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:15:13.266812 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:15:13.266812 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:15:13.266813 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:15:13.267086 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:15:13.267227 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0923 23:15:13.267399 (+ 172us) server_negotiation.cc:299] Negotiation successful
0923 23:15:13.267524 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":95,"threads_started":1}
I20260923 23:15:13.275786 3288 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 23:15:13.276504 3288 catalog_manager.cc:1504] Loading token signing keys...
I20260923 23:15:13.281471 3288 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 05de63bdcc3b4d1cb45d1f00f7427e2c: Generated new TSK 0
I20260923 23:15:13.281666 3288 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.227.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.227.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.227.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.193.keytab.
Entry for principal kudu/127.24.227.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.193.keytab.
I20260923 23:15:13.309022 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790205259943518-25487-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790205259943518-25487-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790205259943518-25487-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790205259943518-25487-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.227.193:0
--local_ip_for_outbound_sockets=127.24.227.193
--webserver_interface=127.24.227.193
--webserver_port=0
--tserver_master_addrs=127.24.227.254:43411
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.193.keytab
--principal=kudu/127.24.227.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:37883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:15:13.387671 3302 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:15:13.387894 3302 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:15:13.387931 3302 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:15:13.389030 3302 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:15:13.389092 3302 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:15:13.389137 3302 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.227.193
I20260923 23:15:13.390563 3302 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:37883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790205259943518-25487-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790205259943518-25487-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.193.keytab
--principal=kudu/127.24.227.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.227.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790205259943518-25487-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.227.193
--webserver_port=0
--tserver_master_addrs=127.24.227.254:43411
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3302
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.227.193
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790205259943518-25487-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:15:13.390800 3302 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:15:13.391018 3302 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:15:13.395550 3307 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:15:13.395569 3310 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:15:13.395558 3308 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:15:13.395807 3302 server_base.cc:1034] running on GCE node
I20260923 23:15:13.396014 3302 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:15:13.396234 3302 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:15:13.397390 3302 hybrid_clock.cc:648] HybridClock initialized: now 1790205313397379 us; error 37 us; skew 500 ppm
Sep 23 23:15:13 dist-test-slave-754n krb5kdc[3195](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205313, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:15:13.399914 3302 init.cc:377] Logged in from keytab as kudu/127.24.227.193@KRBTEST.COM (short username kudu)
I20260923 23:15:13.400278 3302 webserver.cc:460] Webserver started at http://127.24.227.193:43689/ using document root <none> and password file <none>
I20260923 23:15:13.400450 3302 fs_manager.cc:359] Metadata directory not provided
I20260923 23:15:13.400488 3302 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:15:13.400570 3302 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:15:13.401484 3302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790205259943518-25487-0/minicluster-data/ts-0/data/instance:
uuid: "f2b939ab5b75457f9724ffc11f4392d1"
format_stamp: "Formatted at 2026-09-23 23:15:13 on dist-test-slave-754n"
server_key: "a177ba08c765015f0192ff3806833121"
server_key_iv: "103fd01d3e3a7468fad9b5f18218d73b"
server_key_version: "encryptionkey@0"
I20260923 23:15:13.401780 3302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790205259943518-25487-0/minicluster-data/ts-0/wal/instance:
uuid: "f2b939ab5b75457f9724ffc11f4392d1"
format_stamp: "Formatted at 2026-09-23 23:15:13 on dist-test-slave-754n"
server_key: "a177ba08c765015f0192ff3806833121"
server_key_iv: "103fd01d3e3a7468fad9b5f18218d73b"
server_key_version: "encryptionkey@0"
I20260923 23:15:13.402956 3302 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 23:15:13.403623 3317 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:15:13.403772 3302 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 23:15:13.403849 3302 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790205259943518-25487-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790205259943518-25487-0/minicluster-data/ts-0/wal
uuid: "f2b939ab5b75457f9724ffc11f4392d1"
format_stamp: "Formatted at 2026-09-23 23:15:13 on dist-test-slave-754n"
server_key: "a177ba08c765015f0192ff3806833121"
server_key_iv: "103fd01d3e3a7468fad9b5f18218d73b"
server_key_version: "encryptionkey@0"
I20260923 23:15:13.403955 3302 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790205259943518-25487-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790205259943518-25487-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790205259943518-25487-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:15:13.410211 3302 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:15:13.411850 3302 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:15:13.412009 3302 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:15:13.412170 3302 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 23:15:13.412382 3302 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 23:15:13.412410 3302 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 23:15:13.412429 3302 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 23:15:13.412449 3302 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 23:15:13.417951 3302 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.193:43081
I20260923 23:15:13.417999 3430 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.193:43081 every 8 connection(s)
I20260923 23:15:13.418290 3302 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790205259943518-25487-0/minicluster-data/ts-0/data/info.pb
I20260923 23:15:13.423427 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 3302
I20260923 23:15:13.423529 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790205259943518-25487-0/minicluster-data/ts-0/wal/instance
I20260923 23:15:13.423749 25487 external_mini_cluster.cc:1428] Setting key 8b5d9022ed4f2b752bb8d5122ca91b0b
Sep 23 23:15:13 dist-test-slave-754n krb5kdc[3195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205313, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.193@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
I20260923 23:15:13.427342 3279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:15:13.418908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.193:54041 (local address 127.24.227.254:43411)
0923 23:15:13.418965 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0923 23:15:13.418966 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 23:15:13.419132 (+ 166us) server_negotiation.cc:407] Connection header received
0923 23:15:13.419551 (+ 419us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:15:13.419552 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:15:13.419561 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:15:13.419602 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:15:13.420499 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:13.421020 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:13.421910 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:13.422095 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:13.422144 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:13.424385 (+ 2241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:15:13.424386 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:15:13.424386 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:15:13.424393 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:15:13.425947 (+ 1554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:15:13.426433 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:15:13.426433 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:15:13.426434 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:15:13.426484 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:15:13.426732 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:15:13.426733 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:15:13.426733 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:15:13.426825 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:15:13.426868 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0923 23:15:13.427224 (+ 356us) server_negotiation.cc:299] Negotiation successful
0923 23:15:13.427282 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260923 23:15:13.427618 3433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:15:13.418875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:43411 (local address 127.24.227.193:54041)
0923 23:15:13.419018 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0923 23:15:13.419055 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0923 23:15:13.419464 (+ 409us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:15:13.419657 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:15:13.419660 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:15:13.420248 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:15:13.420415 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:15:13.420420 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:15:13.421091 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:15:13.421092 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:15:13.421844 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:15:13.421846 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:15:13.422162 (+ 316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:15:13.422163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:15:13.422256 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:13.422712 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:15:13.422720 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:15:13.424295 (+ 1575us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:15:13.426044 (+ 1749us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:15:13.426051 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:15:13.426053 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:15:13.426359 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:15:13.426540 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:15:13.426541 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:15:13.426542 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:15:13.426669 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:15:13.426888 (+ 219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:15:13.426893 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:15:13.427042 (+ 149us) client_negotiation.cc:769] Sending connection context
0923 23:15:13.427198 (+ 156us) client_negotiation.cc:240] Negotiation successful
0923 23:15:13.427360 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":79,"threads_started":1}
I20260923 23:15:13.428216 3431 heartbeater.cc:347] Connected to a master server at 127.24.227.254:43411
I20260923 23:15:13.428313 3431 heartbeater.cc:464] Registering TS with master...
I20260923 23:15:13.428480 3431 heartbeater.cc:511] Master 127.24.227.254:43411 requested a full tablet report, sending...
I20260923 23:15:13.428911 3240 ts_manager.cc:194] Registered new tserver with Master: f2b939ab5b75457f9724ffc11f4392d1 (127.24.227.193:43081)
I20260923 23:15:13.429535 3240 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.227.193@KRBTEST.COM'} at 127.24.227.193:54041
WARNING: no policy specified for kudu/127.24.227.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.227.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.227.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.194.keytab.
Entry for principal kudu/127.24.227.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.194.keytab.
I20260923 23:15:13.446527 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790205259943518-25487-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790205259943518-25487-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790205259943518-25487-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790205259943518-25487-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.227.194:0
--local_ip_for_outbound_sockets=127.24.227.194
--webserver_interface=127.24.227.194
--webserver_port=0
--tserver_master_addrs=127.24.227.254:43411
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.194.keytab
--principal=kudu/127.24.227.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:37883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:15:13.524256 3436 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:15:13.524430 3436 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:15:13.524451 3436 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:15:13.525478 3436 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:15:13.525511 3436 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:15:13.525535 3436 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.227.194
I20260923 23:15:13.526810 3436 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:37883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790205259943518-25487-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790205259943518-25487-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.194.keytab
--principal=kudu/127.24.227.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.227.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790205259943518-25487-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.227.194
--webserver_port=0
--tserver_master_addrs=127.24.227.254:43411
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3436
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.227.194
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790205259943518-25487-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:15:13.526964 3436 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:15:13.527124 3436 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:15:13.531744 3444 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:15:13.531790 3441 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:15:13.531838 3436 server_base.cc:1034] running on GCE node
W20260923 23:15:13.531769 3442 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:15:13.532076 3436 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:15:13.532305 3436 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:15:13.533447 3436 hybrid_clock.cc:648] HybridClock initialized: now 1790205313533429 us; error 30 us; skew 500 ppm
Sep 23 23:15:13 dist-test-slave-754n krb5kdc[3195](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205313, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:15:13.535779 3436 init.cc:377] Logged in from keytab as kudu/127.24.227.194@KRBTEST.COM (short username kudu)
I20260923 23:15:13.536222 3436 webserver.cc:460] Webserver started at http://127.24.227.194:45669/ using document root <none> and password file <none>
I20260923 23:15:13.536427 3436 fs_manager.cc:359] Metadata directory not provided
I20260923 23:15:13.536487 3436 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:15:13.536618 3436 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:15:13.537585 3436 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790205259943518-25487-0/minicluster-data/ts-1/data/instance:
uuid: "1121d61c9d504ea9aafd0c71a595540a"
format_stamp: "Formatted at 2026-09-23 23:15:13 on dist-test-slave-754n"
server_key: "8d3b170fd94e5d17f5857b2bcb9bd31a"
server_key_iv: "bd2686a909818a31af60e2662703a912"
server_key_version: "encryptionkey@0"
I20260923 23:15:13.537932 3436 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790205259943518-25487-0/minicluster-data/ts-1/wal/instance:
uuid: "1121d61c9d504ea9aafd0c71a595540a"
format_stamp: "Formatted at 2026-09-23 23:15:13 on dist-test-slave-754n"
server_key: "8d3b170fd94e5d17f5857b2bcb9bd31a"
server_key_iv: "bd2686a909818a31af60e2662703a912"
server_key_version: "encryptionkey@0"
I20260923 23:15:13.539234 3436 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 23:15:13.539860 3451 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:15:13.540051 3436 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 23:15:13.540138 3436 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790205259943518-25487-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790205259943518-25487-0/minicluster-data/ts-1/wal
uuid: "1121d61c9d504ea9aafd0c71a595540a"
format_stamp: "Formatted at 2026-09-23 23:15:13 on dist-test-slave-754n"
server_key: "8d3b170fd94e5d17f5857b2bcb9bd31a"
server_key_iv: "bd2686a909818a31af60e2662703a912"
server_key_version: "encryptionkey@0"
I20260923 23:15:13.540218 3436 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790205259943518-25487-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790205259943518-25487-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790205259943518-25487-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:15:13.567559 3436 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:15:13.569810 3436 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:15:13.569962 3436 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:15:13.570174 3436 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 23:15:13.570572 3436 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 23:15:13.570641 3436 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 23:15:13.570678 3436 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 23:15:13.570706 3436 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 23:15:13.576220 3436 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.194:38351
I20260923 23:15:13.576265 3564 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.194:38351 every 8 connection(s)
I20260923 23:15:13.576547 3436 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790205259943518-25487-0/minicluster-data/ts-1/data/info.pb
I20260923 23:15:13.580530 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 3436
I20260923 23:15:13.580658 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790205259943518-25487-0/minicluster-data/ts-1/wal/instance
I20260923 23:15:13.580834 25487 external_mini_cluster.cc:1428] Setting key a7113d25f364773ddfaf5101e1b1f930
Sep 23 23:15:13 dist-test-slave-754n krb5kdc[3195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205313, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.194@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
I20260923 23:15:13.585211 3279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:15:13.577208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.194:53821 (local address 127.24.227.254:43411)
0923 23:15:13.577264 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0923 23:15:13.577266 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:15:13.577438 (+ 172us) server_negotiation.cc:407] Connection header received
0923 23:15:13.577916 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:15:13.577917 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:15:13.577929 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:15:13.577974 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:15:13.578856 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:13.579300 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:13.580281 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:13.580483 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:13.580554 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:13.582628 (+ 2074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:15:13.582629 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:15:13.582629 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:15:13.582637 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:15:13.583907 (+ 1270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:15:13.584296 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:15:13.584297 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:15:13.584298 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:15:13.584372 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:15:13.584618 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:15:13.584619 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:15:13.584619 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:15:13.584734 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:15:13.584804 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0923 23:15:13.585048 (+ 244us) server_negotiation.cc:299] Negotiation successful
0923 23:15:13.585128 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 23:15:13.585440 3567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:15:13.577172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:43411 (local address 127.24.227.194:53821)
0923 23:15:13.577316 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0923 23:15:13.577357 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0923 23:15:13.577817 (+ 460us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:15:13.578048 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:15:13.578052 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:15:13.578594 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:15:13.578766 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:15:13.578771 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:15:13.579385 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:15:13.579386 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:15:13.580193 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:15:13.580195 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:15:13.580557 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:15:13.580558 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:15:13.580705 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:13.581167 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:15:13.581180 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:15:13.582538 (+ 1358us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:15:13.583994 (+ 1456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:15:13.584000 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:15:13.584002 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:15:13.584216 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:15:13.584437 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:15:13.584438 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:15:13.584438 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:15:13.584553 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:15:13.584796 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:15:13.584801 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:15:13.584929 (+ 128us) client_negotiation.cc:769] Sending connection context
0923 23:15:13.585027 (+ 98us) client_negotiation.cc:240] Negotiation successful
0923 23:15:13.585178 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":67,"threads_started":1}
I20260923 23:15:13.585876 3565 heartbeater.cc:347] Connected to a master server at 127.24.227.254:43411
I20260923 23:15:13.585958 3565 heartbeater.cc:464] Registering TS with master...
I20260923 23:15:13.586108 3565 heartbeater.cc:511] Master 127.24.227.254:43411 requested a full tablet report, sending...
I20260923 23:15:13.586452 3240 ts_manager.cc:194] Registered new tserver with Master: 1121d61c9d504ea9aafd0c71a595540a (127.24.227.194:38351)
I20260923 23:15:13.586834 3240 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.227.194@KRBTEST.COM'} at 127.24.227.194:53821
WARNING: no policy specified for kudu/127.24.227.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.227.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.227.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.195.keytab.
Entry for principal kudu/127.24.227.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.195.keytab.
I20260923 23:15:13.603680 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790205259943518-25487-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790205259943518-25487-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790205259943518-25487-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790205259943518-25487-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.227.195:0
--local_ip_for_outbound_sockets=127.24.227.195
--webserver_interface=127.24.227.195
--webserver_port=0
--tserver_master_addrs=127.24.227.254:43411
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.195.keytab
--principal=kudu/127.24.227.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:37883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:15:13.680780 3570 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:15:13.680954 3570 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:15:13.680976 3570 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:15:13.682075 3570 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:15:13.682116 3570 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:15:13.682168 3570 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.227.195
I20260923 23:15:13.683519 3570 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:37883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790205259943518-25487-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790205259943518-25487-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.195.keytab
--principal=kudu/127.24.227.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.227.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790205259943518-25487-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.227.195
--webserver_port=0
--tserver_master_addrs=127.24.227.254:43411
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3570
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.227.195
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790205259943518-25487-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:15:13.683732 3570 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:15:13.683947 3570 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:15:13.688582 3576 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:15:13.688541 3578 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:15:13.688680 3575 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:15:13.688822 3570 server_base.cc:1034] running on GCE node
I20260923 23:15:13.688994 3570 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:15:13.689165 3570 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:15:13.690364 3570 hybrid_clock.cc:648] HybridClock initialized: now 1790205313690359 us; error 54 us; skew 500 ppm
Sep 23 23:15:13 dist-test-slave-754n krb5kdc[3195](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205313, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:15:13.692612 3570 init.cc:377] Logged in from keytab as kudu/127.24.227.195@KRBTEST.COM (short username kudu)
I20260923 23:15:13.692971 3570 webserver.cc:460] Webserver started at http://127.24.227.195:34739/ using document root <none> and password file <none>
I20260923 23:15:13.693127 3570 fs_manager.cc:359] Metadata directory not provided
I20260923 23:15:13.693163 3570 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:15:13.693272 3570 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:15:13.694188 3570 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790205259943518-25487-0/minicluster-data/ts-2/data/instance:
uuid: "0d1956bce3c048d8831b189eb9f05824"
format_stamp: "Formatted at 2026-09-23 23:15:13 on dist-test-slave-754n"
server_key: "a57c59f148379878821841e76a6906a9"
server_key_iv: "777416a8b6c0bb37c9b4bc7ca4ad456e"
server_key_version: "encryptionkey@0"
I20260923 23:15:13.694463 3570 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790205259943518-25487-0/minicluster-data/ts-2/wal/instance:
uuid: "0d1956bce3c048d8831b189eb9f05824"
format_stamp: "Formatted at 2026-09-23 23:15:13 on dist-test-slave-754n"
server_key: "a57c59f148379878821841e76a6906a9"
server_key_iv: "777416a8b6c0bb37c9b4bc7ca4ad456e"
server_key_version: "encryptionkey@0"
I20260923 23:15:13.695547 3570 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 23:15:13.696230 3585 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:15:13.696384 3570 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 23:15:13.696484 3570 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790205259943518-25487-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790205259943518-25487-0/minicluster-data/ts-2/wal
uuid: "0d1956bce3c048d8831b189eb9f05824"
format_stamp: "Formatted at 2026-09-23 23:15:13 on dist-test-slave-754n"
server_key: "a57c59f148379878821841e76a6906a9"
server_key_iv: "777416a8b6c0bb37c9b4bc7ca4ad456e"
server_key_version: "encryptionkey@0"
I20260923 23:15:13.696555 3570 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790205259943518-25487-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790205259943518-25487-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790205259943518-25487-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:15:13.718992 3570 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:15:13.720758 3570 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:15:13.720899 3570 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:15:13.721097 3570 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 23:15:13.721407 3570 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 23:15:13.721457 3570 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 23:15:13.721489 3570 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 23:15:13.721517 3570 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 23:15:13.727043 3570 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.195:44413
I20260923 23:15:13.727074 3698 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.195:44413 every 8 connection(s)
I20260923 23:15:13.727424 3570 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790205259943518-25487-0/minicluster-data/ts-2/data/info.pb
I20260923 23:15:13.727675 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 3570
I20260923 23:15:13.727787 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790205259943518-25487-0/minicluster-data/ts-2/wal/instance
I20260923 23:15:13.728032 25487 external_mini_cluster.cc:1428] Setting key 8f5673db621db252a8326bcd40432c83
Sep 23 23:15:13 dist-test-slave-754n krb5kdc[3195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205313, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.195@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
I20260923 23:15:13.735721 3279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:15:13.728281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.195:60711 (local address 127.24.227.254:43411)
0923 23:15:13.728343 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0923 23:15:13.728345 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:15:13.728566 (+ 221us) server_negotiation.cc:407] Connection header received
0923 23:15:13.729025 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:15:13.729026 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:15:13.729038 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:15:13.729077 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:15:13.729976 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:13.730480 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:13.731393 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:13.731568 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:13.731633 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:13.733526 (+ 1893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:15:13.733527 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:15:13.733528 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:15:13.733534 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:15:13.734692 (+ 1158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:15:13.735018 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:15:13.735019 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:15:13.735019 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:15:13.735065 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:15:13.735275 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:15:13.735275 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:15:13.735276 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:15:13.735366 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:15:13.735413 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0923 23:15:13.735609 (+ 196us) server_negotiation.cc:299] Negotiation successful
0923 23:15:13.735664 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260923 23:15:13.735918 3701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:15:13.728273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:43411 (local address 127.24.227.195:60711)
0923 23:15:13.728436 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0923 23:15:13.728478 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0923 23:15:13.728929 (+ 451us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:15:13.729146 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:15:13.729149 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:15:13.729727 (+ 578us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:15:13.729881 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:15:13.729885 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:15:13.730562 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:15:13.730563 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:15:13.731302 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:15:13.731304 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:15:13.731639 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:15:13.731640 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:15:13.731788 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:13.732215 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:15:13.732223 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:15:13.733447 (+ 1224us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:15:13.734757 (+ 1310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:15:13.734763 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:15:13.734764 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:15:13.734960 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:15:13.735117 (+ 157us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:15:13.735117 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:15:13.735118 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:15:13.735228 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:15:13.735415 (+ 187us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:15:13.735418 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:15:13.735511 (+ 93us) client_negotiation.cc:769] Sending connection context
0923 23:15:13.735584 (+ 73us) client_negotiation.cc:240] Negotiation successful
0923 23:15:13.735687 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":89,"threads_started":1}
I20260923 23:15:13.736415 3699 heartbeater.cc:347] Connected to a master server at 127.24.227.254:43411
I20260923 23:15:13.736531 3699 heartbeater.cc:464] Registering TS with master...
I20260923 23:15:13.736714 3699 heartbeater.cc:511] Master 127.24.227.254:43411 requested a full tablet report, sending...
I20260923 23:15:13.737022 3240 ts_manager.cc:194] Registered new tserver with Master: 0d1956bce3c048d8831b189eb9f05824 (127.24.227.195:44413)
I20260923 23:15:13.737370 3240 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.227.195@KRBTEST.COM'} at 127.24.227.195:60711
I20260923 23:15:13.739413 25487 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 23:15:13.745956 3279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:15:13.740600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45996 (local address 127.24.227.254:43411)
0923 23:15:13.740643 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0923 23:15:13.740645 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:15:13.740748 (+ 103us) server_negotiation.cc:407] Connection header received
0923 23:15:13.740794 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:15:13.740795 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:15:13.740802 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:15:13.740845 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:15:13.741350 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:13.741809 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:13.742495 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:13.742741 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:13.742792 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:13.743788 (+ 996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:15:13.743789 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:15:13.743789 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:15:13.743795 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:15:13.745136 (+ 1341us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:15:13.745418 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:15:13.745419 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:15:13.745419 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:15:13.745462 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:15:13.745574 (+ 112us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:15:13.745574 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:15:13.745575 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:15:13.745690 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:15:13.745757 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0923 23:15:13.745853 (+ 96us) server_negotiation.cc:299] Negotiation successful
0923 23:15:13.745904 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260923 23:15:13.747294 3240 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45996:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 23:15:13.748827 3240 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 23:15:13.754644 3712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:15:13.751139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38188 (local address 127.24.227.195:44413)
0923 23:15:13.751317 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0923 23:15:13.751321 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 23:15:13.751330 (+ 9us) server_negotiation.cc:407] Connection header received
0923 23:15:13.751367 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:15:13.751369 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:15:13.751381 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:15:13.751448 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:15:13.751593 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:13.752404 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:13.753713 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:13.754257 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:13.754377 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:13.754426 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0923 23:15:13.754541 (+ 115us) server_negotiation.cc:299] Negotiation successful
0923 23:15:13.754575 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":89,"threads_started":1}
I20260923 23:15:13.754637 3707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:15:13.750984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.195:44413 (local address 127.0.0.1:38188)
0923 23:15:13.751274 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0923 23:15:13.751283 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 23:15:13.751319 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:15:13.751457 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:15:13.751460 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:15:13.751462 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:15:13.751513 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:15:13.751514 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:15:13.752498 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:15:13.752499 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:15:13.753624 (+ 1125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:15:13.753627 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:15:13.754375 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:15:13.754376 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:15:13.754450 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:13.754462 (+ 12us) client_negotiation.cc:769] Sending connection context
0923 23:15:13.754524 (+ 62us) client_negotiation.cc:240] Negotiation successful
0923 23:15:13.754572 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":58,"threads_started":1}
I20260923 23:15:13.754940 3708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:15:13.750823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.193:43081 (local address 127.0.0.1:46564)
0923 23:15:13.750956 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0923 23:15:13.750989 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0923 23:15:13.751048 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:15:13.751600 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:15:13.751601 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:15:13.751603 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:15:13.751647 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:15:13.751648 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:15:13.752923 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:15:13.752924 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:15:13.753741 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:15:13.753743 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:15:13.754685 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:15:13.754686 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:15:13.754766 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:13.754771 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 23:15:13.754823 (+ 52us) client_negotiation.cc:240] Negotiation successful
0923 23:15:13.754866 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":44,"threads_started":1}
I20260923 23:15:13.754995 3709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:15:13.750934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46564 (local address 127.24.227.193:43081)
0923 23:15:13.751104 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0923 23:15:13.751109 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 23:15:13.751120 (+ 11us) server_negotiation.cc:407] Connection header received
0923 23:15:13.751159 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:15:13.751161 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:15:13.751177 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:15:13.751267 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:15:13.751741 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:13.752840 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:13.753816 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:13.754522 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:13.754593 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:13.754638 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0923 23:15:13.754854 (+ 216us) server_negotiation.cc:299] Negotiation successful
0923 23:15:13.754897 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":71,"threads_started":1}
I20260923 23:15:13.755323 3711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:15:13.750822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.194:38351 (local address 127.0.0.1:48112)
0923 23:15:13.751150 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0923 23:15:13.751159 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 23:15:13.751199 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:15:13.751521 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:15:13.751522 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:15:13.751524 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:15:13.751567 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:15:13.751569 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:15:13.752879 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:15:13.752879 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:15:13.753893 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:15:13.753895 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:15:13.754915 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:15:13.754916 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:15:13.754973 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:13.754977 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 23:15:13.755253 (+ 276us) client_negotiation.cc:240] Negotiation successful
0923 23:15:13.755282 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":309,"mutex_wait_us":2,"thread_start_us":57,"threads_started":1}
I20260923 23:15:13.755359 3633 tablet_service.cc:1467] Processing CreateTablet for tablet 9b652af4d0d1490fa36b75f8a5f8e333 (DEFAULT_TABLE table=test-table [id=aa2de8acddf7446fb6b1e6483eca40f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 23:15:13.755703 3633 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b652af4d0d1490fa36b75f8a5f8e333. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:15:13.755759 3710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:15:13.750933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48112 (local address 127.24.227.194:38351)
0923 23:15:13.751386 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0923 23:15:13.751389 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 23:15:13.751397 (+ 8us) server_negotiation.cc:407] Connection header received
0923 23:15:13.751426 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:15:13.751427 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:15:13.751438 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:15:13.751522 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:15:13.751649 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:13.752764 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:13.753967 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:13.754834 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:13.755527 (+ 693us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:13.755591 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0923 23:15:13.755630 (+ 39us) server_negotiation.cc:299] Negotiation successful
0923 23:15:13.755674 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":424,"thread_start_us":82,"threads_started":1}
I20260923 23:15:13.756428 3499 tablet_service.cc:1467] Processing CreateTablet for tablet 9b652af4d0d1490fa36b75f8a5f8e333 (DEFAULT_TABLE table=test-table [id=aa2de8acddf7446fb6b1e6483eca40f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 23:15:13.756734 3499 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b652af4d0d1490fa36b75f8a5f8e333. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:15:13.758245 3713 tablet_bootstrap.cc:492] T 9b652af4d0d1490fa36b75f8a5f8e333 P 0d1956bce3c048d8831b189eb9f05824: Bootstrap starting.
I20260923 23:15:13.758998 3713 tablet_bootstrap.cc:654] T 9b652af4d0d1490fa36b75f8a5f8e333 P 0d1956bce3c048d8831b189eb9f05824: Neither blocks nor log segments found. Creating new log.
I20260923 23:15:13.758965 3365 tablet_service.cc:1467] Processing CreateTablet for tablet 9b652af4d0d1490fa36b75f8a5f8e333 (DEFAULT_TABLE table=test-table [id=aa2de8acddf7446fb6b1e6483eca40f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 23:15:13.759294 3365 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b652af4d0d1490fa36b75f8a5f8e333. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:15:13.759325 3714 tablet_bootstrap.cc:492] T 9b652af4d0d1490fa36b75f8a5f8e333 P 1121d61c9d504ea9aafd0c71a595540a: Bootstrap starting.
I20260923 23:15:13.759594 3713 log.cc:824] T 9b652af4d0d1490fa36b75f8a5f8e333 P 0d1956bce3c048d8831b189eb9f05824: Log is configured to *not* fsync() on all Append() calls
I20260923 23:15:13.760013 3714 tablet_bootstrap.cc:654] T 9b652af4d0d1490fa36b75f8a5f8e333 P 1121d61c9d504ea9aafd0c71a595540a: Neither blocks nor log segments found. Creating new log.
I20260923 23:15:13.760479 3713 tablet_bootstrap.cc:492] T 9b652af4d0d1490fa36b75f8a5f8e333 P 0d1956bce3c048d8831b189eb9f05824: No bootstrap required, opened a new log
I20260923 23:15:13.760556 3713 ts_tablet_manager.cc:1397] T 9b652af4d0d1490fa36b75f8a5f8e333 P 0d1956bce3c048d8831b189eb9f05824: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260923 23:15:13.760597 3714 log.cc:824] T 9b652af4d0d1490fa36b75f8a5f8e333 P 1121d61c9d504ea9aafd0c71a595540a: Log is configured to *not* fsync() on all Append() calls
I20260923 23:15:13.761368 3714 tablet_bootstrap.cc:492] T 9b652af4d0d1490fa36b75f8a5f8e333 P 1121d61c9d504ea9aafd0c71a595540a: No bootstrap required, opened a new log
I20260923 23:15:13.761439 3714 ts_tablet_manager.cc:1397] T 9b652af4d0d1490fa36b75f8a5f8e333 P 1121d61c9d504ea9aafd0c71a595540a: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260923 23:15:13.762171 3717 tablet_bootstrap.cc:492] T 9b652af4d0d1490fa36b75f8a5f8e333 P f2b939ab5b75457f9724ffc11f4392d1: Bootstrap starting.
I20260923 23:15:13.762565 3713 raft_consensus.cc:348] T 9b652af4d0d1490fa36b75f8a5f8e333 P 0d1956bce3c048d8831b189eb9f05824 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1121d61c9d504ea9aafd0c71a595540a" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 38351 } } peers { permanent_uuid: "f2b939ab5b75457f9724ffc11f4392d1" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 43081 } } peers { permanent_uuid: "0d1956bce3c048d8831b189eb9f05824" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 44413 } }
I20260923 23:15:13.762733 3713 raft_consensus.cc:374] T 9b652af4d0d1490fa36b75f8a5f8e333 P 0d1956bce3c048d8831b189eb9f05824 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:15:13.762763 3717 tablet_bootstrap.cc:654] T 9b652af4d0d1490fa36b75f8a5f8e333 P f2b939ab5b75457f9724ffc11f4392d1: Neither blocks nor log segments found. Creating new log.
I20260923 23:15:13.762769 3713 raft_consensus.cc:729] T 9b652af4d0d1490fa36b75f8a5f8e333 P 0d1956bce3c048d8831b189eb9f05824 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d1956bce3c048d8831b189eb9f05824, State: Initialized, Role: FOLLOWER
I20260923 23:15:13.762908 3713 consensus_queue.cc:260] T 9b652af4d0d1490fa36b75f8a5f8e333 P 0d1956bce3c048d8831b189eb9f05824 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1121d61c9d504ea9aafd0c71a595540a" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 38351 } } peers { permanent_uuid: "f2b939ab5b75457f9724ffc11f4392d1" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 43081 } } peers { permanent_uuid: "0d1956bce3c048d8831b189eb9f05824" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 44413 } }
I20260923 23:15:13.763186 3713 ts_tablet_manager.cc:1428] T 9b652af4d0d1490fa36b75f8a5f8e333 P 0d1956bce3c048d8831b189eb9f05824: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 23:15:13.763329 3699 heartbeater.cc:503] Master 127.24.227.254:43411 was elected leader, sending a full tablet report...
I20260923 23:15:13.763427 3717 log.cc:824] T 9b652af4d0d1490fa36b75f8a5f8e333 P f2b939ab5b75457f9724ffc11f4392d1: Log is configured to *not* fsync() on all Append() calls
I20260923 23:15:13.763958 3714 raft_consensus.cc:348] T 9b652af4d0d1490fa36b75f8a5f8e333 P 1121d61c9d504ea9aafd0c71a595540a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1121d61c9d504ea9aafd0c71a595540a" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 38351 } } peers { permanent_uuid: "f2b939ab5b75457f9724ffc11f4392d1" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 43081 } } peers { permanent_uuid: "0d1956bce3c048d8831b189eb9f05824" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 44413 } }
I20260923 23:15:13.764125 3714 raft_consensus.cc:374] T 9b652af4d0d1490fa36b75f8a5f8e333 P 1121d61c9d504ea9aafd0c71a595540a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:15:13.764174 3714 raft_consensus.cc:729] T 9b652af4d0d1490fa36b75f8a5f8e333 P 1121d61c9d504ea9aafd0c71a595540a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1121d61c9d504ea9aafd0c71a595540a, State: Initialized, Role: FOLLOWER
I20260923 23:15:13.764346 3717 tablet_bootstrap.cc:492] T 9b652af4d0d1490fa36b75f8a5f8e333 P f2b939ab5b75457f9724ffc11f4392d1: No bootstrap required, opened a new log
I20260923 23:15:13.764456 3717 ts_tablet_manager.cc:1397] T 9b652af4d0d1490fa36b75f8a5f8e333 P f2b939ab5b75457f9724ffc11f4392d1: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260923 23:15:13.764299 3714 consensus_queue.cc:260] T 9b652af4d0d1490fa36b75f8a5f8e333 P 1121d61c9d504ea9aafd0c71a595540a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1121d61c9d504ea9aafd0c71a595540a" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 38351 } } peers { permanent_uuid: "f2b939ab5b75457f9724ffc11f4392d1" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 43081 } } peers { permanent_uuid: "0d1956bce3c048d8831b189eb9f05824" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 44413 } }
I20260923 23:15:13.764725 3714 ts_tablet_manager.cc:1428] T 9b652af4d0d1490fa36b75f8a5f8e333 P 1121d61c9d504ea9aafd0c71a595540a: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260923 23:15:13.764739 3565 heartbeater.cc:503] Master 127.24.227.254:43411 was elected leader, sending a full tablet report...
I20260923 23:15:13.765826 3717 raft_consensus.cc:348] T 9b652af4d0d1490fa36b75f8a5f8e333 P f2b939ab5b75457f9724ffc11f4392d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1121d61c9d504ea9aafd0c71a595540a" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 38351 } } peers { permanent_uuid: "f2b939ab5b75457f9724ffc11f4392d1" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 43081 } } peers { permanent_uuid: "0d1956bce3c048d8831b189eb9f05824" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 44413 } }
I20260923 23:15:13.765972 3717 raft_consensus.cc:374] T 9b652af4d0d1490fa36b75f8a5f8e333 P f2b939ab5b75457f9724ffc11f4392d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:15:13.766034 3717 raft_consensus.cc:729] T 9b652af4d0d1490fa36b75f8a5f8e333 P f2b939ab5b75457f9724ffc11f4392d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2b939ab5b75457f9724ffc11f4392d1, State: Initialized, Role: FOLLOWER
I20260923 23:15:13.766136 3717 consensus_queue.cc:260] T 9b652af4d0d1490fa36b75f8a5f8e333 P f2b939ab5b75457f9724ffc11f4392d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1121d61c9d504ea9aafd0c71a595540a" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 38351 } } peers { permanent_uuid: "f2b939ab5b75457f9724ffc11f4392d1" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 43081 } } peers { permanent_uuid: "0d1956bce3c048d8831b189eb9f05824" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 44413 } }
I20260923 23:15:13.766399 3431 heartbeater.cc:503] Master 127.24.227.254:43411 was elected leader, sending a full tablet report...
I20260923 23:15:13.766409 3717 ts_tablet_manager.cc:1428] T 9b652af4d0d1490fa36b75f8a5f8e333 P f2b939ab5b75457f9724ffc11f4392d1: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
W20260923 23:15:13.827126 3566 tablet.cc:2369] T 9b652af4d0d1490fa36b75f8a5f8e333 P 1121d61c9d504ea9aafd0c71a595540a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 23:15:13.918979 3432 tablet.cc:2369] T 9b652af4d0d1490fa36b75f8a5f8e333 P f2b939ab5b75457f9724ffc11f4392d1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 23:15:13.977981 3700 tablet.cc:2369] T 9b652af4d0d1490fa36b75f8a5f8e333 P 0d1956bce3c048d8831b189eb9f05824: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 23:15:14.173877 3718 raft_consensus.cc:482] T 9b652af4d0d1490fa36b75f8a5f8e333 P 0d1956bce3c048d8831b189eb9f05824 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 23:15:14.174019 3718 raft_consensus.cc:504] T 9b652af4d0d1490fa36b75f8a5f8e333 P 0d1956bce3c048d8831b189eb9f05824 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1121d61c9d504ea9aafd0c71a595540a" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 38351 } } peers { permanent_uuid: "f2b939ab5b75457f9724ffc11f4392d1" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 43081 } } peers { permanent_uuid: "0d1956bce3c048d8831b189eb9f05824" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 44413 } }
I20260923 23:15:14.174360 3718 leader_election.cc:288] T 9b652af4d0d1490fa36b75f8a5f8e333 P 0d1956bce3c048d8831b189eb9f05824 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1121d61c9d504ea9aafd0c71a595540a (127.24.227.194:38351), f2b939ab5b75457f9724ffc11f4392d1 (127.24.227.193:43081)
I20260923 23:15:14.177371 3710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:15:14.174596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.195:36531 (local address 127.24.227.194:38351)
0923 23:15:14.174675 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0923 23:15:14.174678 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 23:15:14.174721 (+ 43us) server_negotiation.cc:407] Connection header received
0923 23:15:14.174752 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:15:14.174752 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:15:14.174761 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:15:14.174812 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:15:14.175033 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:14.175583 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:14.176681 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:14.177025 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:14.177080 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:14.177086 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0923 23:15:14.177256 (+ 170us) server_negotiation.cc:299] Negotiation successful
0923 23:15:14.177295 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260923 23:15:14.177376 3709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:15:14.174627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.195:57313 (local address 127.24.227.193:43081)
0923 23:15:14.174689 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0923 23:15:14.174690 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 23:15:14.174924 (+ 234us) server_negotiation.cc:407] Connection header received
0923 23:15:14.174953 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:15:14.174954 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:15:14.174967 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:15:14.175040 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:15:14.175229 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:14.175725 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:14.176733 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:14.177052 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:14.177138 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:14.177145 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0923 23:15:14.177282 (+ 137us) server_negotiation.cc:299] Negotiation successful
0923 23:15:14.177317 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 23:15:14.177373 3701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:15:14.174547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.194:38351 (local address 127.24.227.195:36531)
0923 23:15:14.174596 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0923 23:15:14.174625 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0923 23:15:14.174680 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:15:14.174840 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:15:14.174841 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:15:14.174843 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:15:14.174938 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:15:14.174940 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:15:14.175658 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:15:14.175659 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:15:14.176573 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:15:14.176575 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:15:14.177094 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:15:14.177096 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:15:14.177174 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:14.177177 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 23:15:14.177233 (+ 56us) client_negotiation.cc:240] Negotiation successful
0923 23:15:14.177286 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29,"mutex_wait_us":1}
I20260923 23:15:14.177387 3722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:15:14.174552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.193:43081 (local address 127.24.227.195:57313)
0923 23:15:14.174854 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0923 23:15:14.174863 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 23:15:14.174913 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:15:14.175055 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:15:14.175056 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:15:14.175058 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:15:14.175119 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:15:14.175121 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:15:14.175802 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:15:14.175803 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:15:14.176587 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:15:14.176588 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:15:14.177146 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:15:14.177147 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:15:14.177197 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:14.177200 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 23:15:14.177268 (+ 68us) client_negotiation.cc:240] Negotiation successful
0923 23:15:14.177302 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"mutex_wait_us":168,"thread_start_us":99,"threads_started":1}
I20260923 23:15:14.177701 3385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b652af4d0d1490fa36b75f8a5f8e333" candidate_uuid: "0d1956bce3c048d8831b189eb9f05824" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b939ab5b75457f9724ffc11f4392d1" is_pre_election: true
I20260923 23:15:14.177740 3519 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b652af4d0d1490fa36b75f8a5f8e333" candidate_uuid: "0d1956bce3c048d8831b189eb9f05824" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1121d61c9d504ea9aafd0c71a595540a" is_pre_election: true
I20260923 23:15:14.177845 3385 raft_consensus.cc:2454] T 9b652af4d0d1490fa36b75f8a5f8e333 P f2b939ab5b75457f9724ffc11f4392d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d1956bce3c048d8831b189eb9f05824 in term 0.
I20260923 23:15:14.177911 3519 raft_consensus.cc:2454] T 9b652af4d0d1490fa36b75f8a5f8e333 P 1121d61c9d504ea9aafd0c71a595540a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d1956bce3c048d8831b189eb9f05824 in term 0.
I20260923 23:15:14.178068 3586 leader_election.cc:302] T 9b652af4d0d1490fa36b75f8a5f8e333 P 0d1956bce3c048d8831b189eb9f05824 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d1956bce3c048d8831b189eb9f05824, f2b939ab5b75457f9724ffc11f4392d1; no voters:
I20260923 23:15:14.178187 3718 raft_consensus.cc:2781] T 9b652af4d0d1490fa36b75f8a5f8e333 P 0d1956bce3c048d8831b189eb9f05824 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 23:15:14.178256 3718 raft_consensus.cc:482] T 9b652af4d0d1490fa36b75f8a5f8e333 P 0d1956bce3c048d8831b189eb9f05824 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 23:15:14.178284 3718 raft_consensus.cc:3037] T 9b652af4d0d1490fa36b75f8a5f8e333 P 0d1956bce3c048d8831b189eb9f05824 [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:15:14.178951 3718 raft_consensus.cc:504] T 9b652af4d0d1490fa36b75f8a5f8e333 P 0d1956bce3c048d8831b189eb9f05824 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1121d61c9d504ea9aafd0c71a595540a" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 38351 } } peers { permanent_uuid: "f2b939ab5b75457f9724ffc11f4392d1" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 43081 } } peers { permanent_uuid: "0d1956bce3c048d8831b189eb9f05824" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 44413 } }
I20260923 23:15:14.179116 3718 leader_election.cc:288] T 9b652af4d0d1490fa36b75f8a5f8e333 P 0d1956bce3c048d8831b189eb9f05824 [CANDIDATE]: Term 1 election: Requested vote from peers 1121d61c9d504ea9aafd0c71a595540a (127.24.227.194:38351), f2b939ab5b75457f9724ffc11f4392d1 (127.24.227.193:43081)
I20260923 23:15:14.179282 3385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b652af4d0d1490fa36b75f8a5f8e333" candidate_uuid: "0d1956bce3c048d8831b189eb9f05824" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b939ab5b75457f9724ffc11f4392d1"
I20260923 23:15:14.179294 3519 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b652af4d0d1490fa36b75f8a5f8e333" candidate_uuid: "0d1956bce3c048d8831b189eb9f05824" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1121d61c9d504ea9aafd0c71a595540a"
I20260923 23:15:14.179355 3519 raft_consensus.cc:3037] T 9b652af4d0d1490fa36b75f8a5f8e333 P 1121d61c9d504ea9aafd0c71a595540a [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:15:14.179389 3385 raft_consensus.cc:3037] T 9b652af4d0d1490fa36b75f8a5f8e333 P f2b939ab5b75457f9724ffc11f4392d1 [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:15:14.180145 3519 raft_consensus.cc:2454] T 9b652af4d0d1490fa36b75f8a5f8e333 P 1121d61c9d504ea9aafd0c71a595540a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d1956bce3c048d8831b189eb9f05824 in term 1.
I20260923 23:15:14.180255 3385 raft_consensus.cc:2454] T 9b652af4d0d1490fa36b75f8a5f8e333 P f2b939ab5b75457f9724ffc11f4392d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d1956bce3c048d8831b189eb9f05824 in term 1.
I20260923 23:15:14.180310 3588 leader_election.cc:302] T 9b652af4d0d1490fa36b75f8a5f8e333 P 0d1956bce3c048d8831b189eb9f05824 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d1956bce3c048d8831b189eb9f05824, 1121d61c9d504ea9aafd0c71a595540a; no voters:
I20260923 23:15:14.180434 3718 raft_consensus.cc:2781] T 9b652af4d0d1490fa36b75f8a5f8e333 P 0d1956bce3c048d8831b189eb9f05824 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 23:15:14.180596 3718 raft_consensus.cc:686] T 9b652af4d0d1490fa36b75f8a5f8e333 P 0d1956bce3c048d8831b189eb9f05824 [term 1 LEADER]: Becoming Leader. State: Replica: 0d1956bce3c048d8831b189eb9f05824, State: Running, Role: LEADER
I20260923 23:15:14.180691 3718 consensus_queue.cc:237] T 9b652af4d0d1490fa36b75f8a5f8e333 P 0d1956bce3c048d8831b189eb9f05824 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1121d61c9d504ea9aafd0c71a595540a" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 38351 } } peers { permanent_uuid: "f2b939ab5b75457f9724ffc11f4392d1" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 43081 } } peers { permanent_uuid: "0d1956bce3c048d8831b189eb9f05824" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 44413 } }
I20260923 23:15:14.181538 3240 catalog_manager.cc:5495] T 9b652af4d0d1490fa36b75f8a5f8e333 P 0d1956bce3c048d8831b189eb9f05824 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0d1956bce3c048d8831b189eb9f05824 (127.24.227.195). New cstate: current_term: 1 leader_uuid: "0d1956bce3c048d8831b189eb9f05824" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1121d61c9d504ea9aafd0c71a595540a" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 38351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b939ab5b75457f9724ffc11f4392d1" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 43081 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d1956bce3c048d8831b189eb9f05824" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 44413 } health_report { overall_health: HEALTHY } } }
I20260923 23:15:14.191576 3712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:15:14.189299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38192 (local address 127.24.227.195:44413)
0923 23:15:14.189345 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0923 23:15:14.189347 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:15:14.189355 (+ 8us) server_negotiation.cc:407] Connection header received
0923 23:15:14.189422 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:15:14.189422 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:15:14.189438 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:15:14.189516 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 23:15:14.189654 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:14.190109 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:14.190846 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:14.191025 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:14.191101 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:14.191272 (+ 171us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 23:15:14.191364 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 23:15:14.191441 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0923 23:15:14.191466 (+ 25us) server_negotiation.cc:299] Negotiation successful
0923 23:15:14.191484 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260923 23:15:14.193148 3385 raft_consensus.cc:1261] T 9b652af4d0d1490fa36b75f8a5f8e333 P f2b939ab5b75457f9724ffc11f4392d1 [term 1 FOLLOWER]: Refusing update from remote peer 0d1956bce3c048d8831b189eb9f05824: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 23:15:14.193171 3519 raft_consensus.cc:1261] T 9b652af4d0d1490fa36b75f8a5f8e333 P 1121d61c9d504ea9aafd0c71a595540a [term 1 FOLLOWER]: Refusing update from remote peer 0d1956bce3c048d8831b189eb9f05824: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 23:15:14.193485 3718 consensus_queue.cc:1035] T 9b652af4d0d1490fa36b75f8a5f8e333 P 0d1956bce3c048d8831b189eb9f05824 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1121d61c9d504ea9aafd0c71a595540a" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 38351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 23:15:14.193604 3723 consensus_queue.cc:1035] T 9b652af4d0d1490fa36b75f8a5f8e333 P 0d1956bce3c048d8831b189eb9f05824 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b939ab5b75457f9724ffc11f4392d1" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 43081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 23:15:14.195806 3726 mvcc.cc:205] Tried to move back new op lower bound from 7332680966932492288 to 7332680966884458496. Current Snapshot: MvccSnapshot[applied={T|T < 7332680966932492288}]
I20260923 23:15:14.196050 3727 mvcc.cc:205] Tried to move back new op lower bound from 7332680966932492288 to 7332680966884458496. Current Snapshot: MvccSnapshot[applied={T|T < 7332680966932492288}]
I20260923 23:15:14.196112 3728 mvcc.cc:205] Tried to move back new op lower bound from 7332680966932492288 to 7332680966884458496. Current Snapshot: MvccSnapshot[applied={T|T < 7332680966932492288}]
I20260923 23:15:14.198297 3238 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45996:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 23:15:14.198385 3238 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45996:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 23:15:14.199123 3238 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 05de63bdcc3b4d1cb45d1f00f7427e2c: Sending DeleteTablet for 3 replicas of tablet 9b652af4d0d1490fa36b75f8a5f8e333
I20260923 23:15:14.199491 3499 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b652af4d0d1490fa36b75f8a5f8e333 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 23:15:14 UTC) from {username='kudu', principal='kudu/127.24.227.254@KRBTEST.COM'} at 127.0.0.1:48112
I20260923 23:15:14.199733 3736 tablet_replica.cc:330] T 9b652af4d0d1490fa36b75f8a5f8e333 P 1121d61c9d504ea9aafd0c71a595540a: stopping tablet replica
I20260923 23:15:14.199718 3365 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b652af4d0d1490fa36b75f8a5f8e333 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 23:15:14 UTC) from {username='kudu', principal='kudu/127.24.227.254@KRBTEST.COM'} at 127.0.0.1:46564
I20260923 23:15:14.199843 3736 raft_consensus.cc:2229] T 9b652af4d0d1490fa36b75f8a5f8e333 P 1121d61c9d504ea9aafd0c71a595540a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 23:15:14.199975 3736 raft_consensus.cc:2258] T 9b652af4d0d1490fa36b75f8a5f8e333 P 1121d61c9d504ea9aafd0c71a595540a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 23:15:14.200059 3633 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b652af4d0d1490fa36b75f8a5f8e333 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 23:15:14 UTC) from {username='kudu', principal='kudu/127.24.227.254@KRBTEST.COM'} at 127.0.0.1:38188
I20260923 23:15:14.200223 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 3302
I20260923 23:15:14.200340 3737 tablet_replica.cc:330] T 9b652af4d0d1490fa36b75f8a5f8e333 P f2b939ab5b75457f9724ffc11f4392d1: stopping tablet replica
I20260923 23:15:14.200397 3736 ts_tablet_manager.cc:1905] T 9b652af4d0d1490fa36b75f8a5f8e333 P 1121d61c9d504ea9aafd0c71a595540a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 23:15:14.200491 3737 raft_consensus.cc:2229] T 9b652af4d0d1490fa36b75f8a5f8e333 P f2b939ab5b75457f9724ffc11f4392d1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 23:15:14.200608 3737 raft_consensus.cc:2258] T 9b652af4d0d1490fa36b75f8a5f8e333 P f2b939ab5b75457f9724ffc11f4392d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 23:15:14.202466 3736 ts_tablet_manager.cc:1918] T 9b652af4d0d1490fa36b75f8a5f8e333 P 1121d61c9d504ea9aafd0c71a595540a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 23:15:14.202539 3736 log.cc:1196] T 9b652af4d0d1490fa36b75f8a5f8e333 P 1121d61c9d504ea9aafd0c71a595540a: Deleting WAL directory at /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790205259943518-25487-0/minicluster-data/ts-1/wal/wals/9b652af4d0d1490fa36b75f8a5f8e333
I20260923 23:15:14.202809 3736 ts_tablet_manager.cc:1939] T 9b652af4d0d1490fa36b75f8a5f8e333 P 1121d61c9d504ea9aafd0c71a595540a: Deleting consensus metadata
I20260923 23:15:14.203222 3226 catalog_manager.cc:4841] TS 1121d61c9d504ea9aafd0c71a595540a (127.24.227.194:38351): tablet 9b652af4d0d1490fa36b75f8a5f8e333 (table test-table [id=aa2de8acddf7446fb6b1e6483eca40f6]) successfully deleted
W20260923 23:15:14.207814 3224 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.227.193:43081 (error 108)
I20260923 23:15:14.207842 3738 tablet_replica.cc:330] T 9b652af4d0d1490fa36b75f8a5f8e333 P 0d1956bce3c048d8831b189eb9f05824: stopping tablet replica
I20260923 23:15:14.208009 3738 raft_consensus.cc:2229] T 9b652af4d0d1490fa36b75f8a5f8e333 P 0d1956bce3c048d8831b189eb9f05824 [term 1 LEADER]: Raft consensus shutting down.
I20260923 23:15:14.208105 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 3436
I20260923 23:15:14.208143 3738 raft_consensus.cc:2258] T 9b652af4d0d1490fa36b75f8a5f8e333 P 0d1956bce3c048d8831b189eb9f05824 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 23:15:14.209657 3707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:15:14.209516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.193:43081 (local address 127.0.0.1:46578)
0923 23:15:14.209578 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0923 23:15:14.209623 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.193:43081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260923 23:15:14.209769 3224 catalog_manager.cc:4584] TS f2b939ab5b75457f9724ffc11f4392d1 (127.24.227.193:43081): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9b652af4d0d1490fa36b75f8a5f8e333: Network error: Client connection negotiation failed: client connection to 127.24.227.193:43081: connect: Connection refused (error 111)
I20260923 23:15:14.209949 3738 ts_tablet_manager.cc:1905] T 9b652af4d0d1490fa36b75f8a5f8e333 P 0d1956bce3c048d8831b189eb9f05824: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 23:15:14.211903 3738 ts_tablet_manager.cc:1918] T 9b652af4d0d1490fa36b75f8a5f8e333 P 0d1956bce3c048d8831b189eb9f05824: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 23:15:14.211978 3738 log.cc:1196] T 9b652af4d0d1490fa36b75f8a5f8e333 P 0d1956bce3c048d8831b189eb9f05824: Deleting WAL directory at /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790205259943518-25487-0/minicluster-data/ts-2/wal/wals/9b652af4d0d1490fa36b75f8a5f8e333
I20260923 23:15:14.212224 3738 ts_tablet_manager.cc:1939] T 9b652af4d0d1490fa36b75f8a5f8e333 P 0d1956bce3c048d8831b189eb9f05824: Deleting consensus metadata
I20260923 23:15:14.212625 3224 catalog_manager.cc:4841] TS 0d1956bce3c048d8831b189eb9f05824 (127.24.227.195:44413): tablet 9b652af4d0d1490fa36b75f8a5f8e333 (table test-table [id=aa2de8acddf7446fb6b1e6483eca40f6]) successfully deleted
I20260923 23:15:14.214172 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 3570
I20260923 23:15:14.218678 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 3208
2026-09-23T23:15:14Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2217 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 23:15:14 dist-test-slave-754n krb5kdc[3742](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 23:15:14 dist-test-slave-754n krb5kdc[3742](info): set up 2 sockets
Sep 23 23:15:14 dist-test-slave-754n krb5kdc[3742](info): commencing operation
krb5kdc: starting...
W20260923 23:15:15.228703 25487 mini_kdc.cc:121] Time spent starting KDC: real 0.990s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/krb5kdc/test-user.keytab.
Sep 23 23:15:15 dist-test-slave-754n krb5kdc[3742](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205315, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T23:15:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T23:15:15Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.24.227.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.24.227.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.24.227.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/krb5kdc/rangeradmin_127.24.227.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.24.227.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/krb5kdc/rangeradmin_127.24.227.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.24.227.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.24.227.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.24.227.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/krb5kdc/rangerlookup_127.24.227.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.24.227.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/krb5kdc/rangerlookup_127.24.227.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.24.227.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.24.227.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.24.227.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/krb5kdc/HTTP_127.24.227.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.24.227.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/krb5kdc/HTTP_127.24.227.212@KRBTEST.COM.keytab.
I20260923 23:15:15.380613 25487 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskyHY_dZ/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/minicluster-data/postgres -l logfile start
2026-09-23 23:15:17.681 UTC [3772] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-23 23:15:17.681 UTC [3772] LOG: listening on IPv4 address "127.24.227.212", port 35499
2026-09-23 23:15:17.684 UTC [3772] LOG: listening on Unix socket "/tmp/.s.PGSQL.35499"
2026-09-23 23:15:17.718 UTC [3775] LOG: database system was shut down at 2026-09-23 23:15:16 UTC
2026-09-23 23:15:17.721 UTC [3772] LOG: database system is ready to accept connections
I20260923 23:15:18.660669 25487 mini_postgres.cc:96] Postgres bound to 35499
2026-09-23 23:15:18.666 UTC [3783] FATAL: database "slave" does not exist
127.24.227.212:35499 - accepting connections
I20260923 23:15:18.667011 25487 mini_ranger.cc:161] Starting Ranger...
I20260923 23:15:18.680315 25487 mini_ranger.cc:84] Created miniranger Postgres user
I20260923 23:15:19.090364 25487 mini_ranger.cc:87] Created ranger Postgres database
I20260923 23:15:19.090476 25487 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/minicluster-data/ranger-admin
2026-09-23 23:15:19,353 [I] DB FLAVOR :POSTGRES
2026-09-23 23:15:19,353 [I] --------- Verifying Ranger DB connection ---------
2026-09-23 23:15:19,353 [I] Checking connection..
2026-09-23 23:15:19,353 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyHY_dZ/build/release/bin/postgresql.jar:/tmp/dist-test-taskyHY_dZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.227.212:35499/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-23 23:15:19,595 [I] Checking connection passed.
2026-09-23 23:15:19,595 [I] --------- Verifying version history table ---------
2026-09-23 23:15:19,595 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyHY_dZ/build/release/bin/postgresql.jar:/tmp/dist-test-taskyHY_dZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.227.212:35499/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-23 23:15:19,811 [I] Table x_db_version_h does not exist in database ranger
2026-09-23 23:15:19,811 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyHY_dZ/build/release/bin/postgresql.jar:/tmp/dist-test-taskyHY_dZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.227.212:35499/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-23 23:15:20,036 [I] Table x_db_version_h does not exist in database ranger
2026-09-23 23:15:20,037 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-23 23:15:20,037 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyHY_dZ/build/release/bin/postgresql.jar:/tmp/dist-test-taskyHY_dZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.227.212:35499/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskyHY_dZ/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-23 23:15:20.283 UTC [3847] WARNING: there is no transaction in progress
2026-09-23 23:15:20,288 [I] create_dbversion_catalog.sql file imported successfully
2026-09-23 23:15:20,289 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyHY_dZ/build/release/bin/postgresql.jar:/tmp/dist-test-taskyHY_dZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.227.212:35499/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-23 23:15:20,518 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-23 23:15:20,518 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-23 23:15:20,518 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyHY_dZ/build/release/bin/postgresql.jar:/tmp/dist-test-taskyHY_dZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.227.212:35499/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-23 23:15:20,760 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyHY_dZ/build/release/bin/postgresql.jar:/tmp/dist-test-taskyHY_dZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.227.212:35499/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-23 23:15:20,983 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyHY_dZ/build/release/bin/postgresql.jar:/tmp/dist-test-taskyHY_dZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.227.212:35499/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-754n.c.gcp-upstream.internal','N') ;"
2026-09-23 23:15:21,226 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyHY_dZ/build/release/bin/postgresql.jar:/tmp/dist-test-taskyHY_dZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.227.212:35499/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-23 23:15:21,465 [I] Table x_portal_user does not exist in database ranger
2026-09-23 23:15:21,465 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyHY_dZ/build/release/bin/postgresql.jar:/tmp/dist-test-taskyHY_dZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.227.212:35499/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-23 23:15:21,700 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-23 23:15:21,700 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyHY_dZ/build/release/bin/postgresql.jar:/tmp/dist-test-taskyHY_dZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.227.212:35499/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-23 23:15:21,919 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyHY_dZ/build/release/bin/postgresql.jar:/tmp/dist-test-taskyHY_dZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.227.212:35499/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-23 23:15:22,166 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-23 23:15:22,167 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyHY_dZ/build/release/bin/postgresql.jar:/tmp/dist-test-taskyHY_dZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.227.212:35499/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskyHY_dZ/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-23 23:15:23.025 UTC [3955] WARNING: there is no transaction in progress
2026-09-23 23:15:23.041 UTC [3955] WARNING: there is no transaction in progress
2026-09-23 23:15:23.056 UTC [3955] WARNING: there is no transaction in progress
2026-09-23 23:15:23.071 UTC [3955] WARNING: there is no transaction in progress
2026-09-23 23:15:23.088 UTC [3955] WARNING: there is no transaction in progress
2026-09-23 23:15:23.107 UTC [3955] WARNING: there is no transaction in progress
2026-09-23 23:15:23.161 UTC [3955] WARNING: there is no transaction in progress
2026-09-23 23:15:23.167 UTC [3955] WARNING: there is no transaction in progress
2026-09-23 23:15:23.181 UTC [3955] WARNING: there is no transaction in progress
2026-09-23 23:15:23.191 UTC [3955] WARNING: there is no transaction in progress
2026-09-23 23:15:23.200 UTC [3955] WARNING: there is no transaction in progress
2026-09-23 23:15:23.212 UTC [3955] WARNING: there is no transaction in progress
2026-09-23 23:15:23.221 UTC [3955] WARNING: there is no transaction in progress
2026-09-23 23:15:23.226 UTC [3955] WARNING: there is no transaction in progress
2026-09-23 23:15:23.232 UTC [3955] WARNING: there is no transaction in progress
2026-09-23 23:15:23.239 UTC [3955] WARNING: there is no transaction in progress
2026-09-23 23:15:23,804 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-23 23:15:23,804 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyHY_dZ/build/release/bin/postgresql.jar:/tmp/dist-test-taskyHY_dZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.227.212:35499/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-23 23:15:24,014 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-23 23:15:24,015 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyHY_dZ/build/release/bin/postgresql.jar:/tmp/dist-test-taskyHY_dZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.227.212:35499/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-23 23:15:24,250 [I] Table x_portal_user already exists in database 'ranger'
2026-09-23 23:15:24,250 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyHY_dZ/build/release/bin/postgresql.jar:/tmp/dist-test-taskyHY_dZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.227.212:35499/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-23 23:15:24,472 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-23 23:15:24,472 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyHY_dZ/build/release/bin/postgresql.jar:/tmp/dist-test-taskyHY_dZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.227.212:35499/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-23 23:15:24,698 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyHY_dZ/build/release/bin/postgresql.jar:/tmp/dist-test-taskyHY_dZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.227.212:35499/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-23 23:15:24,914 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyHY_dZ/build/release/bin/postgresql.jar:/tmp/dist-test-taskyHY_dZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.227.212:35499/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-754n.c.gcp-upstream.internal';"
2026-09-23 23:15:25,140 [I] CORE_DB_SCHEMA import status has been updated
2026-09-23 23:15:25,141 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyHY_dZ/build/release/bin/postgresql.jar:/tmp/dist-test-taskyHY_dZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.227.212:35499/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-23 23:15:25,354 [I] DB_PATCHES have already been applied
I20260923 23:15:25.359583 25487 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/minicluster-data/ranger-admin:/tmp/dist-test-taskyHY_dZ/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskyHY_dZ/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskyHY_dZ/build/release/bin/postgresql.jar:/tmp/dist-test-taskyHY_dZ/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260923 23:15:25.359638 25487 mini_ranger.cc:193] Using host: 127.24.227.212
I20260923 23:15:25.362587 25487 mini_ranger.cc:227] Ranger admin URL: http://127.24.227.212:33081
Sep 23, 2026 11:15:25 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 23, 2026 11:15:25 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 23, 2026 11:15:25 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/minicluster-data/ranger-admin/ews/webapp
Sep 23, 2026 11:15:25 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 23, 2026 11:15:25 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 23, 2026 11:15:26 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/minicluster-data/ranger-admin/ews/webapp].
Sep 23, 2026 11:15:26 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260923 23:15:26.430215 25487 mini_ranger.cc:160] Time spent starting Ranger: real 7.763s user 0.000s sys 0.008s
Sep 23, 2026 11:15:45 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 23, 2026 11:15:45 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Sep 23, 2026 11:15:45 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 23, 2026 11:15:45 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 23, 2026 11:15:45 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 23, 2026 11:15:45 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 23, 2026 11:15:45 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 23, 2026 11:15:45 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 23, 2026 11:15:45 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 23, 2026 11:15:45 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 23, 2026 11:15:45 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 23, 2026 11:15:45 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 23, 2026 11:15:45 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 23, 2026 11:15:45 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 23, 2026 11:15:45 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 23, 2026 11:15:45 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 23, 2026 11:15:45 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 23, 2026 11:15:46 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260923 23:15:47.389803 25487 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.24.227.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.24.227.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.24.227.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/krb5kdc/rangerkms_127.24.227.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.24.227.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/krb5kdc/rangerkms_127.24.227.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.24.227.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.24.227.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.24.227.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/krb5kdc/HTTP_127.24.227.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.24.227.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/krb5kdc/HTTP_127.24.227.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 23 23:15:47 dist-test-slave-754n krb5kdc[3742](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205347, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260923 23:15:47.464880 25487 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260923 23:15:47.473723 25487 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260923 23:15:48.255908 25487 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260923 23:15:48.255987 25487 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/minicluster-data/ranger-kms
I20260923 23:15:48.256001 25487 mini_ranger_kms.cc:227] Using postgres at 127.24.227.212:35499
2026-09-23 23:15:48,428 [I] DB FLAVOR :POSTGRES
2026-09-23 23:15:48,429 [I] --------- Verifying Ranger DB connection ---------
2026-09-23 23:15:48,429 [I] Checking connection
2026-09-23 23:15:48,429 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyHY_dZ/build/release/bin/postgresql.jar:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.227.212:35499/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-23 23:15:48,662 [I] connection success
2026-09-23 23:15:48,663 [I] --------- Verifying Ranger DB tables ---------
2026-09-23 23:15:48,663 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-23 23:15:48,663 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyHY_dZ/build/release/bin/postgresql.jar:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.227.212:35499/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-23 23:15:48,907 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-23 23:15:48,907 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-23 23:15:48,907 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-23 23:15:48,908 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyHY_dZ/build/release/bin/postgresql.jar:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.227.212:35499/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-23 23:15:49,175 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260923 23:15:49.617002 25487 mini_ranger_kms.cc:310] Created kms service
I20260923 23:15:49.779691 25487 mini_ranger_kms.cc:326] Created kudu user
I20260923 23:15:49.827378 25487 mini_ranger_kms.cc:343] Created rangerkms user
Sep 23 23:15:49 dist-test-slave-754n krb5kdc[3742](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205347, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.24.227.212@KRBTEST.COM
I20260923 23:15:50.210445 25487 mini_ranger_kms.cc:384] Added ranger policy
I20260923 23:15:50.210729 25487 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/minicluster-data/ranger-kms:/tmp/dist-test-taskyHY_dZ/build/release/bin/postgresql.jar:/tmp/dist-test-taskyHY_dZ/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskyHY_dZ/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260923 23:15:50.210772 25487 mini_ranger_kms.cc:240] Using host: 127.24.227.212
I20260923 23:15:50.214265 25487 mini_ranger_kms.cc:276] Ranger KMS PID: 4135
I20260923 23:15:50.214357 25487 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.24.227.212:37369
Sep 23, 2026 11:15:51 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 23, 2026 11:15:51 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 23, 2026 11:15:51 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 23, 2026 11:15:51 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 23, 2026 11:15:51 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/minicluster-data/ranger-kms/ews/webapp].
Sep 23, 2026 11:15:51 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260923 23:15:52.419343 25487 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 4.954s user 0.000s sys 0.156s
I20260923 23:15:52.419432 25487 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260923 23:15:52.419517 25487 mini_ranger_kms.cc:401] 127.24.227.212:37369/kms/v1/keys
Sep 23 23:15:52 dist-test-slave-754n krb5kdc[3742](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790205352, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.24.227.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 23 23:15:53 dist-test-slave-754n krb5kdc[3742](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205352, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.24.227.212@KRBTEST.COM for HTTP/127.24.227.212@KRBTEST.COM
Sep 23 23:15:53 dist-test-slave-754n krb5kdc[3742](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205352, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.24.227.212@KRBTEST.COM for HTTP/127.24.227.212@KRBTEST.COM
Sep 23 23:15:55 dist-test-slave-754n krb5kdc[3742](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205355, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.24.227.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.227.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.227.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab.
Entry for principal kudu/127.24.227.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab.
I20260923 23:15:55.634519 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.254:44363
--webserver_interface=127.24.227.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab
--principal=kudu/127.24.227.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:37917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.227.254:44363
--ranger_config_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.227.212:37369/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:15:55.713191 4186 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:15:55.713400 4186 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:15:55.713436 4186 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:15:55.714541 4186 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 23:15:55.714646 4186 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:15:55.714669 4186 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:15:55.714689 4186 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 23:15:55.714718 4186 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 23:15:55.716253 4186 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:37917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/minicluster-data/master-0/wal
--ranger_kms_url=127.24.227.212:37369/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.24.227.254:44363
--ranger_config_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab
--principal=kudu/127.24.227.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.254:44363
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.227.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4186
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:15:55.716500 4186 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:15:55.716713 4186 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:15:55.722292 4192 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:15:55.722353 4194 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:15:55.722288 4191 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:15:55.722548 4186 server_base.cc:1034] running on GCE node
I20260923 23:15:55.722766 4186 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:15:55.723081 4186 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:15:55.724236 4186 hybrid_clock.cc:648] HybridClock initialized: now 1790205355724205 us; error 48 us; skew 500 ppm
Sep 23 23:15:55 dist-test-slave-754n krb5kdc[3742](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205355, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:15:55.731104 4186 init.cc:377] Logged in from keytab as kudu/127.24.227.254@KRBTEST.COM (short username kudu)
I20260923 23:15:55.731560 4186 webserver.cc:460] Webserver started at http://127.24.227.254:46319/ using document root <none> and password file <none>
I20260923 23:15:55.731829 4186 fs_manager.cc:359] Metadata directory not provided
I20260923 23:15:55.731911 4186 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:15:55.732060 4186 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 23 23:15:55 dist-test-slave-754n krb5kdc[3742](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205355, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.254@KRBTEST.COM for HTTP/127.24.227.212@KRBTEST.COM
I20260923 23:15:55.781435 4186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/minicluster-data/master-0/data/instance:
uuid: "bb36251ef7cb4a3eb3f41d0813b7de1f"
format_stamp: "Formatted at 2026-09-23 23:15:55 on dist-test-slave-754n"
server_key: "80c56b90d254d7bdc5dccaaced0ae396"
server_key_iv: "061578cd5f009bff35977b581afe23fd"
server_key_version: "kuduclusterkey@0"
I20260923 23:15:55.781838 4186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/minicluster-data/master-0/wal/instance:
uuid: "bb36251ef7cb4a3eb3f41d0813b7de1f"
format_stamp: "Formatted at 2026-09-23 23:15:55 on dist-test-slave-754n"
server_key: "80c56b90d254d7bdc5dccaaced0ae396"
server_key_iv: "061578cd5f009bff35977b581afe23fd"
server_key_version: "kuduclusterkey@0"
I20260923 23:15:55.783387 4186 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260923 23:15:55.794116 4201 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:15:55.794413 4186 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 23:15:55.794505 4186 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/minicluster-data/master-0/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/minicluster-data/master-0/wal
uuid: "bb36251ef7cb4a3eb3f41d0813b7de1f"
format_stamp: "Formatted at 2026-09-23 23:15:55 on dist-test-slave-754n"
server_key: "80c56b90d254d7bdc5dccaaced0ae396"
server_key_iv: "061578cd5f009bff35977b581afe23fd"
server_key_version: "kuduclusterkey@0"
I20260923 23:15:55.794580 4186 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:15:55.807417 4186 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:15:55.809284 4186 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:15:55.809438 4186 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:15:55.813977 4186 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.254:44363
I20260923 23:15:55.814129 4253 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.254:44363 every 8 connection(s)
I20260923 23:15:55.814529 4186 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
I20260923 23:15:55.815286 4254 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:15:55.818167 4254 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb36251ef7cb4a3eb3f41d0813b7de1f: Bootstrap starting.
I20260923 23:15:55.818841 4254 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bb36251ef7cb4a3eb3f41d0813b7de1f: Neither blocks nor log segments found. Creating new log.
I20260923 23:15:55.819156 4254 log.cc:824] T 00000000000000000000000000000000 P bb36251ef7cb4a3eb3f41d0813b7de1f: Log is configured to *not* fsync() on all Append() calls
I20260923 23:15:55.819341 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 4186
I20260923 23:15:55.819455 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/minicluster-data/master-0/wal/instance
I20260923 23:15:55.819684 25487 external_mini_cluster.cc:1428] Setting key aaef41baf87efd97eff6e086c720c9bc
I20260923 23:15:55.819969 4254 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb36251ef7cb4a3eb3f41d0813b7de1f: No bootstrap required, opened a new log
I20260923 23:15:55.822839 4254 raft_consensus.cc:348] T 00000000000000000000000000000000 P bb36251ef7cb4a3eb3f41d0813b7de1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb36251ef7cb4a3eb3f41d0813b7de1f" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 44363 } }
I20260923 23:15:55.823017 4254 raft_consensus.cc:374] T 00000000000000000000000000000000 P bb36251ef7cb4a3eb3f41d0813b7de1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:15:55.823048 4254 raft_consensus.cc:729] T 00000000000000000000000000000000 P bb36251ef7cb4a3eb3f41d0813b7de1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb36251ef7cb4a3eb3f41d0813b7de1f, State: Initialized, Role: FOLLOWER
I20260923 23:15:55.823206 4254 consensus_queue.cc:260] T 00000000000000000000000000000000 P bb36251ef7cb4a3eb3f41d0813b7de1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb36251ef7cb4a3eb3f41d0813b7de1f" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 44363 } }
I20260923 23:15:55.823273 4254 raft_consensus.cc:388] T 00000000000000000000000000000000 P bb36251ef7cb4a3eb3f41d0813b7de1f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 23:15:55.823300 4254 raft_consensus.cc:482] T 00000000000000000000000000000000 P bb36251ef7cb4a3eb3f41d0813b7de1f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 23:15:55.823370 4254 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bb36251ef7cb4a3eb3f41d0813b7de1f [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:15:55.823985 4254 raft_consensus.cc:504] T 00000000000000000000000000000000 P bb36251ef7cb4a3eb3f41d0813b7de1f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb36251ef7cb4a3eb3f41d0813b7de1f" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 44363 } }
I20260923 23:15:55.824082 4254 leader_election.cc:302] T 00000000000000000000000000000000 P bb36251ef7cb4a3eb3f41d0813b7de1f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bb36251ef7cb4a3eb3f41d0813b7de1f; no voters:
I20260923 23:15:55.824211 4254 leader_election.cc:288] T 00000000000000000000000000000000 P bb36251ef7cb4a3eb3f41d0813b7de1f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 23:15:55.824257 4259 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bb36251ef7cb4a3eb3f41d0813b7de1f [term 1 FOLLOWER]: Leader election won for term 1
I20260923 23:15:55.824533 4254 sys_catalog.cc:564] T 00000000000000000000000000000000 P bb36251ef7cb4a3eb3f41d0813b7de1f [sys.catalog]: configured and running, proceeding with master startup.
I20260923 23:15:55.824754 4259 raft_consensus.cc:686] T 00000000000000000000000000000000 P bb36251ef7cb4a3eb3f41d0813b7de1f [term 1 LEADER]: Becoming Leader. State: Replica: bb36251ef7cb4a3eb3f41d0813b7de1f, State: Running, Role: LEADER
I20260923 23:15:55.824890 4259 consensus_queue.cc:237] T 00000000000000000000000000000000 P bb36251ef7cb4a3eb3f41d0813b7de1f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb36251ef7cb4a3eb3f41d0813b7de1f" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 44363 } }
I20260923 23:15:55.825209 4259 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb36251ef7cb4a3eb3f41d0813b7de1f [sys.catalog]: SysCatalogTable state changed. Reason: New leader bb36251ef7cb4a3eb3f41d0813b7de1f. Latest consensus state: current_term: 1 leader_uuid: "bb36251ef7cb4a3eb3f41d0813b7de1f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb36251ef7cb4a3eb3f41d0813b7de1f" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 44363 } } }
I20260923 23:15:55.825294 4259 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb36251ef7cb4a3eb3f41d0813b7de1f [sys.catalog]: This master's current role is: LEADER
I20260923 23:15:55.826169 4254 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260923 23:15:55.826248 4261 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb36251ef7cb4a3eb3f41d0813b7de1f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bb36251ef7cb4a3eb3f41d0813b7de1f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb36251ef7cb4a3eb3f41d0813b7de1f" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 44363 } } }
I20260923 23:15:55.826388 4261 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb36251ef7cb4a3eb3f41d0813b7de1f [sys.catalog]: This master's current role is: LEADER
Sep 23 23:15:55 dist-test-slave-754n krb5kdc[3742](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205355, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
I20260923 23:15:55.830298 4267 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 23:15:55.834208 4258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:15:55.820170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33676 (local address 127.24.227.254:44363)
0923 23:15:55.820405 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0923 23:15:55.820413 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 23:15:55.820436 (+ 23us) server_negotiation.cc:407] Connection header received
0923 23:15:55.821243 (+ 807us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:15:55.821249 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:15:55.821293 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:15:55.821432 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:15:55.822030 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:55.822872 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:55.823802 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:55.824355 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:55.824463 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:55.829589 (+ 5126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:15:55.829650 (+ 61us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:15:55.829651 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:15:55.829765 (+ 114us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:15:55.832487 (+ 2722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:15:55.832937 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:15:55.832940 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:15:55.832940 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:15:55.833039 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:15:55.833267 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:15:55.833267 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:15:55.833268 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:15:55.833570 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:15:55.833720 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0923 23:15:55.833851 (+ 131us) server_negotiation.cc:299] Negotiation successful
0923 23:15:55.833997 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":75,"threads_started":1}
I20260923 23:15:55.834481 4267 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 23:15:55.836221 4267 catalog_manager.cc:1347] Generated new cluster ID: afd2ad7bf13c40558906c8a4b919d855
I20260923 23:15:55.836270 4267 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 23:15:55.848616 4267 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 23:15:55.849344 4267 catalog_manager.cc:1504] Loading token signing keys...
I20260923 23:15:55.855312 4267 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bb36251ef7cb4a3eb3f41d0813b7de1f: Generated new TSK 0
I20260923 23:15:55.855509 4267 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.227.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.227.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.227.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.193.keytab.
Entry for principal kudu/127.24.227.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.193.keytab.
I20260923 23:15:56.586165 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.227.212:37369/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.227.193:0
--local_ip_for_outbound_sockets=127.24.227.193
--webserver_interface=127.24.227.193
--webserver_port=0
--tserver_master_addrs=127.24.227.254:44363
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.193.keytab
--principal=kudu/127.24.227.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:37917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:15:56.676085 4299 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:15:56.676256 4299 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:15:56.676278 4299 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:15:56.677376 4299 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:15:56.677414 4299 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:15:56.677457 4299 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.227.193
I20260923 23:15:56.678833 4299 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:37917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.24.227.212:37369/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.193.keytab
--principal=kudu/127.24.227.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.227.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.227.193
--webserver_port=0
--tserver_master_addrs=127.24.227.254:44363
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4299
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.227.193
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:15:56.679037 4299 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:15:56.679268 4299 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:15:56.684114 4305 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:15:56.684129 4304 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:15:56.684108 4307 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:15:56.684379 4299 server_base.cc:1034] running on GCE node
I20260923 23:15:56.684551 4299 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:15:56.684787 4299 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:15:56.685945 4299 hybrid_clock.cc:648] HybridClock initialized: now 1790205356685935 us; error 25 us; skew 500 ppm
Sep 23 23:15:56 dist-test-slave-754n krb5kdc[3742](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205356, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:15:56.688305 4299 init.cc:377] Logged in from keytab as kudu/127.24.227.193@KRBTEST.COM (short username kudu)
I20260923 23:15:56.688683 4299 webserver.cc:460] Webserver started at http://127.24.227.193:43963/ using document root <none> and password file <none>
I20260923 23:15:56.688915 4299 fs_manager.cc:359] Metadata directory not provided
I20260923 23:15:56.688982 4299 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:15:56.689109 4299 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 23 23:15:56 dist-test-slave-754n krb5kdc[3742](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205356, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.193@KRBTEST.COM for HTTP/127.24.227.212@KRBTEST.COM
I20260923 23:15:56.701293 4299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/minicluster-data/ts-0/data/instance:
uuid: "1649bed544794bfa8321d452f7d518e5"
format_stamp: "Formatted at 2026-09-23 23:15:56 on dist-test-slave-754n"
server_key: "b44cf700b00f9e0914c784d117c65e97"
server_key_iv: "35d59477b55debf6588481914652dd22"
server_key_version: "kuduclusterkey@0"
I20260923 23:15:56.701720 4299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/minicluster-data/ts-0/wal/instance:
uuid: "1649bed544794bfa8321d452f7d518e5"
format_stamp: "Formatted at 2026-09-23 23:15:56 on dist-test-slave-754n"
server_key: "b44cf700b00f9e0914c784d117c65e97"
server_key_iv: "35d59477b55debf6588481914652dd22"
server_key_version: "kuduclusterkey@0"
I20260923 23:15:56.703464 4299 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20260923 23:15:56.715222 4314 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:15:56.715415 4299 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 23:15:56.715503 4299 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/minicluster-data/ts-0/wal
uuid: "1649bed544794bfa8321d452f7d518e5"
format_stamp: "Formatted at 2026-09-23 23:15:56 on dist-test-slave-754n"
server_key: "b44cf700b00f9e0914c784d117c65e97"
server_key_iv: "35d59477b55debf6588481914652dd22"
server_key_version: "kuduclusterkey@0"
I20260923 23:15:56.715579 4299 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:15:56.747239 4299 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:15:56.749621 4299 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:15:56.749826 4299 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:15:56.750151 4299 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 23:15:56.750551 4299 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 23:15:56.750640 4299 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 23:15:56.750736 4299 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 23:15:56.750808 4299 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 23:15:56.758350 4299 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.193:43451
I20260923 23:15:56.758822 4299 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/minicluster-data/ts-0/data/info.pb
I20260923 23:15:56.759341 4427 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.193:43451 every 8 connection(s)
I20260923 23:15:56.761657 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 4299
I20260923 23:15:56.761758 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/minicluster-data/ts-0/wal/instance
I20260923 23:15:56.761965 25487 external_mini_cluster.cc:1428] Setting key 9e66dd2a9a25b4233eedaefb3dec74bd
Sep 23 23:15:56 dist-test-slave-754n krb5kdc[3742](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205356, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.193@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
I20260923 23:15:56.775856 4430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:15:56.759628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.193:52543 (local address 127.24.227.254:44363)
0923 23:15:56.759804 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0923 23:15:56.759806 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:15:56.760102 (+ 296us) server_negotiation.cc:407] Connection header received
0923 23:15:56.760740 (+ 638us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:15:56.760741 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:15:56.760758 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:15:56.760841 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:15:56.764172 (+ 3331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:56.764958 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:56.766465 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:56.766758 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:56.766839 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:56.769320 (+ 2481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:15:56.769322 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:15:56.769322 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:15:56.769329 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:15:56.771196 (+ 1867us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:15:56.771690 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:15:56.771691 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:15:56.771692 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:15:56.771758 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:15:56.772046 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:15:56.772047 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:15:56.772047 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:15:56.772180 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:15:56.772234 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0923 23:15:56.775658 (+ 3424us) server_negotiation.cc:299] Negotiation successful
0923 23:15:56.775747 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":95,"threads_started":1}
I20260923 23:15:56.776623 4431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:15:56.759635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:44363 (local address 127.24.227.193:52543)
0923 23:15:56.759964 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0923 23:15:56.760007 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0923 23:15:56.760618 (+ 611us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:15:56.760921 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:15:56.760926 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:15:56.762412 (+ 1486us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:15:56.764082 (+ 1670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:15:56.764088 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:15:56.765081 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:15:56.765082 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:15:56.766366 (+ 1284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:15:56.766369 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:15:56.766858 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:15:56.766859 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:15:56.767000 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:56.767575 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:15:56.767584 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:15:56.769207 (+ 1623us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:15:56.771293 (+ 2086us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:15:56.771299 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:15:56.771301 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:15:56.771598 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:15:56.771824 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:15:56.771825 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:15:56.771825 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:15:56.771970 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:15:56.772266 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:15:56.772271 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:15:56.772447 (+ 176us) client_negotiation.cc:769] Sending connection context
0923 23:15:56.776223 (+ 3776us) client_negotiation.cc:240] Negotiation successful
0923 23:15:56.776382 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":88,"threads_started":1}
I20260923 23:15:56.777616 4428 heartbeater.cc:347] Connected to a master server at 127.24.227.254:44363
I20260923 23:15:56.777714 4428 heartbeater.cc:464] Registering TS with master...
I20260923 23:15:56.777906 4428 heartbeater.cc:511] Master 127.24.227.254:44363 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.24.227.194@KRBTEST.COM; defaulting to no policy
I20260923 23:15:56.778872 4218 ts_manager.cc:194] Registered new tserver with Master: 1649bed544794bfa8321d452f7d518e5 (127.24.227.193:43451)
I20260923 23:15:56.779757 4218 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.227.193@KRBTEST.COM'} at 127.24.227.193:52543
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.227.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.227.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.194.keytab.
Entry for principal kudu/127.24.227.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.194.keytab.
I20260923 23:15:56.800753 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.227.212:37369/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.227.194:0
--local_ip_for_outbound_sockets=127.24.227.194
--webserver_interface=127.24.227.194
--webserver_port=0
--tserver_master_addrs=127.24.227.254:44363
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.194.keytab
--principal=kudu/127.24.227.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:37917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:15:56.912833 4436 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:15:56.913051 4436 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:15:56.913087 4436 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:15:56.914242 4436 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:15:56.914335 4436 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:15:56.914381 4436 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.227.194
I20260923 23:15:56.915938 4436 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:37917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.24.227.212:37369/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.194.keytab
--principal=kudu/127.24.227.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.227.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.227.194
--webserver_port=0
--tserver_master_addrs=127.24.227.254:44363
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4436
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.227.194
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:15:56.916175 4436 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:15:56.916400 4436 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:15:56.921216 4444 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:15:56.921257 4446 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:15:56.921581 4443 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:15:56.922124 4436 server_base.cc:1034] running on GCE node
I20260923 23:15:56.922320 4436 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:15:56.922597 4436 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:15:56.923745 4436 hybrid_clock.cc:648] HybridClock initialized: now 1790205356923732 us; error 41 us; skew 500 ppm
Sep 23 23:15:56 dist-test-slave-754n krb5kdc[3742](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205356, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:15:56.926963 4436 init.cc:377] Logged in from keytab as kudu/127.24.227.194@KRBTEST.COM (short username kudu)
I20260923 23:15:56.927434 4436 webserver.cc:460] Webserver started at http://127.24.227.194:45325/ using document root <none> and password file <none>
I20260923 23:15:56.927661 4436 fs_manager.cc:359] Metadata directory not provided
I20260923 23:15:56.927727 4436 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:15:56.927858 4436 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 23 23:15:56 dist-test-slave-754n krb5kdc[3742](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205356, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.194@KRBTEST.COM for HTTP/127.24.227.212@KRBTEST.COM
I20260923 23:15:56.938565 4436 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/minicluster-data/ts-1/data/instance:
uuid: "490732101ced4f2fa34f753d6706da1c"
format_stamp: "Formatted at 2026-09-23 23:15:56 on dist-test-slave-754n"
server_key: "5e1d047ab4d2d2b1e295f608528e5571"
server_key_iv: "c37e70727552af04a8a99001646c1839"
server_key_version: "kuduclusterkey@0"
I20260923 23:15:56.939033 4436 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/minicluster-data/ts-1/wal/instance:
uuid: "490732101ced4f2fa34f753d6706da1c"
format_stamp: "Formatted at 2026-09-23 23:15:56 on dist-test-slave-754n"
server_key: "5e1d047ab4d2d2b1e295f608528e5571"
server_key_iv: "c37e70727552af04a8a99001646c1839"
server_key_version: "kuduclusterkey@0"
I20260923 23:15:56.944739 4436 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.001s
I20260923 23:15:56.959992 4453 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:15:56.960194 4436 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.001s
I20260923 23:15:56.960312 4436 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/minicluster-data/ts-1/wal
uuid: "490732101ced4f2fa34f753d6706da1c"
format_stamp: "Formatted at 2026-09-23 23:15:56 on dist-test-slave-754n"
server_key: "5e1d047ab4d2d2b1e295f608528e5571"
server_key_iv: "c37e70727552af04a8a99001646c1839"
server_key_version: "kuduclusterkey@0"
I20260923 23:15:56.960438 4436 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
Sep 23 23:15:56 dist-test-slave-754n krb5kdc[3742](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790205356, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:15:56.983139 4436 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:15:56.986413 4436 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:15:56.986599 4436 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:15:56.986845 4436 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 23:15:56.987251 4436 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 23:15:56.987347 4436 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 23:15:56.987417 4436 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 23:15:56.987474 4436 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 23:15:56.994366 4436 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.194:35449
I20260923 23:15:56.994794 4436 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/minicluster-data/ts-1/data/info.pb
I20260923 23:15:56.996579 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 4436
I20260923 23:15:56.996701 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/minicluster-data/ts-1/wal/instance
I20260923 23:15:56.996913 25487 external_mini_cluster.cc:1428] Setting key 74372e509ef8f89bc8bfdc2278a47f5b
I20260923 23:15:56.996497 4566 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.194:35449 every 8 connection(s)
Sep 23 23:15:57 dist-test-slave-754n krb5kdc[3742](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205356, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.194@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
I20260923 23:15:57.020362 4430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:15:56.998099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.194:52771 (local address 127.24.227.254:44363)
0923 23:15:56.998158 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0923 23:15:56.998160 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:15:57.008079 (+ 9919us) server_negotiation.cc:407] Connection header received
0923 23:15:57.008924 (+ 845us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:15:57.008925 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:15:57.008942 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:15:57.008974 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:15:57.009798 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:57.010585 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:57.011548 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:57.011902 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:57.011969 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:57.015924 (+ 3955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:15:57.015925 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:15:57.015925 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:15:57.015934 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:15:57.017587 (+ 1653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:15:57.018081 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:15:57.018082 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:15:57.018083 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:15:57.018142 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:15:57.018413 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:15:57.018414 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:15:57.018414 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:15:57.018568 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:15:57.018624 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0923 23:15:57.020162 (+ 1538us) server_negotiation.cc:299] Negotiation successful
0923 23:15:57.020253 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260923 23:15:57.021100 4569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:15:56.997854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:44363 (local address 127.24.227.194:52771)
0923 23:15:57.007935 (+ 10081us) negotiation.cc:107] Waiting for socket to connect
0923 23:15:57.007983 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0923 23:15:57.008657 (+ 674us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:15:57.009172 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:15:57.009176 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:15:57.009416 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:15:57.009678 (+ 262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:15:57.009683 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:15:57.010681 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:15:57.010682 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:15:57.011455 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:15:57.011457 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:15:57.012036 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:15:57.012038 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:15:57.012184 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:57.012713 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:15:57.012722 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:15:57.015783 (+ 3061us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:15:57.017692 (+ 1909us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:15:57.017698 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:15:57.017700 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:15:57.017971 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:15:57.018214 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:15:57.018215 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:15:57.018215 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:15:57.018339 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:15:57.018675 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:15:57.018681 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:15:57.018847 (+ 166us) client_negotiation.cc:769] Sending connection context
0923 23:15:57.020732 (+ 1885us) client_negotiation.cc:240] Negotiation successful
0923 23:15:57.020883 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10036,"thread_start_us":79,"threads_started":1}
I20260923 23:15:57.021735 4567 heartbeater.cc:347] Connected to a master server at 127.24.227.254:44363
I20260923 23:15:57.021822 4567 heartbeater.cc:464] Registering TS with master...
I20260923 23:15:57.022001 4567 heartbeater.cc:511] Master 127.24.227.254:44363 requested a full tablet report, sending...
I20260923 23:15:57.022415 4218 ts_manager.cc:194] Registered new tserver with Master: 490732101ced4f2fa34f753d6706da1c (127.24.227.194:35449)
I20260923 23:15:57.022924 4218 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.227.194@KRBTEST.COM'} at 127.24.227.194:52771
WARNING: no policy specified for kudu/127.24.227.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.227.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.227.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.195.keytab.
Entry for principal kudu/127.24.227.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.195.keytab.
I20260923 23:15:57.042629 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.227.212:37369/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.227.195:0
--local_ip_for_outbound_sockets=127.24.227.195
--webserver_interface=127.24.227.195
--webserver_port=0
--tserver_master_addrs=127.24.227.254:44363
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.195.keytab
--principal=kudu/127.24.227.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:37917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:15:57.159272 4572 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:15:57.159592 4572 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:15:57.159699 4572 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:15:57.161413 4572 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:15:57.161520 4572 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:15:57.161598 4572 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.227.195
I20260923 23:15:57.163677 4572 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:37917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.24.227.212:37369/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.195.keytab
--principal=kudu/127.24.227.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.227.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.227.195
--webserver_port=0
--tserver_master_addrs=127.24.227.254:44363
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4572
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.227.195
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:15:57.164222 4572 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:15:57.164541 4572 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:15:57.170871 4579 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:15:57.170853 4581 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:15:57.171666 4572 server_base.cc:1034] running on GCE node
W20260923 23:15:57.172035 4578 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:15:57.172353 4572 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:15:57.172596 4572 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:15:57.177053 4572 hybrid_clock.cc:648] HybridClock initialized: now 1790205357177031 us; error 35 us; skew 500 ppm
Sep 23 23:15:57 dist-test-slave-754n krb5kdc[3742](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205357, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:15:57.180215 4572 init.cc:377] Logged in from keytab as kudu/127.24.227.195@KRBTEST.COM (short username kudu)
I20260923 23:15:57.180724 4572 webserver.cc:460] Webserver started at http://127.24.227.195:45119/ using document root <none> and password file <none>
I20260923 23:15:57.180971 4572 fs_manager.cc:359] Metadata directory not provided
I20260923 23:15:57.181017 4572 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:15:57.181129 4572 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 23 23:15:57 dist-test-slave-754n krb5kdc[3742](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205357, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.195@KRBTEST.COM for HTTP/127.24.227.212@KRBTEST.COM
I20260923 23:15:57.193701 4572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/minicluster-data/ts-2/data/instance:
uuid: "fe67dff8bc4e406187a56a2c48bb5ac7"
format_stamp: "Formatted at 2026-09-23 23:15:57 on dist-test-slave-754n"
server_key: "d7a6e40002b1be79afb4eee9188efe96"
server_key_iv: "610be0338481574c0bea12b5f80a008c"
server_key_version: "kuduclusterkey@0"
I20260923 23:15:57.194186 4572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/minicluster-data/ts-2/wal/instance:
uuid: "fe67dff8bc4e406187a56a2c48bb5ac7"
format_stamp: "Formatted at 2026-09-23 23:15:57 on dist-test-slave-754n"
server_key: "d7a6e40002b1be79afb4eee9188efe96"
server_key_iv: "610be0338481574c0bea12b5f80a008c"
server_key_version: "kuduclusterkey@0"
I20260923 23:15:57.195847 4572 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 23:15:57.221462 4588 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:15:57.221635 4572 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260923 23:15:57.221771 4572 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/minicluster-data/ts-2/wal
uuid: "fe67dff8bc4e406187a56a2c48bb5ac7"
format_stamp: "Formatted at 2026-09-23 23:15:57 on dist-test-slave-754n"
server_key: "d7a6e40002b1be79afb4eee9188efe96"
server_key_iv: "610be0338481574c0bea12b5f80a008c"
server_key_version: "kuduclusterkey@0"
I20260923 23:15:57.221868 4572 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:15:57.244659 4572 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:15:57.249893 4572 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:15:57.250067 4572 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:15:57.250299 4572 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 23:15:57.250670 4572 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 23:15:57.250723 4572 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 23:15:57.250758 4572 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 23:15:57.250785 4572 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 23:15:57.257463 4572 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.195:34811
I20260923 23:15:57.257786 4701 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.195:34811 every 8 connection(s)
I20260923 23:15:57.257943 4572 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/minicluster-data/ts-2/data/info.pb
I20260923 23:15:57.259684 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 4572
I20260923 23:15:57.259786 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/minicluster-data/ts-2/wal/instance
I20260923 23:15:57.259984 25487 external_mini_cluster.cc:1428] Setting key fd8cce2a289b9453859ec4c332a4d4bc
Sep 23 23:15:57 dist-test-slave-754n krb5kdc[3742](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205357, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.195@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
I20260923 23:15:57.269423 4430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:15:57.258656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.195:44463 (local address 127.24.227.254:44363)
0923 23:15:57.258891 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0923 23:15:57.258894 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 23:15:57.259473 (+ 579us) server_negotiation.cc:407] Connection header received
0923 23:15:57.260368 (+ 895us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:15:57.260369 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:15:57.260381 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:15:57.260454 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:15:57.261211 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:57.261997 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:57.263478 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:57.263724 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:57.263812 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:57.266266 (+ 2454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:15:57.266267 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:15:57.266268 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:15:57.266274 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:15:57.267834 (+ 1560us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:15:57.268325 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:15:57.268326 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:15:57.268326 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:15:57.268384 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:15:57.268691 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:15:57.268691 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:15:57.268692 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:15:57.268817 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:15:57.268883 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0923 23:15:57.269245 (+ 362us) server_negotiation.cc:299] Negotiation successful
0923 23:15:57.269330 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":205}
I20260923 23:15:57.269943 4704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:15:57.258654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:44363 (local address 127.24.227.195:44463)
0923 23:15:57.259340 (+ 686us) negotiation.cc:107] Waiting for socket to connect
0923 23:15:57.259387 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0923 23:15:57.260235 (+ 848us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:15:57.260586 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:15:57.260591 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:15:57.260813 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:15:57.261095 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:15:57.261102 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:15:57.262105 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:15:57.262106 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:15:57.263383 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:15:57.263385 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:15:57.263819 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:15:57.263821 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:15:57.263977 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:57.264495 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:15:57.264504 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:15:57.266163 (+ 1659us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:15:57.267950 (+ 1787us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:15:57.267956 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:15:57.267958 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:15:57.268248 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:15:57.268460 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:15:57.268461 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:15:57.268461 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:15:57.268616 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:15:57.268896 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:15:57.268902 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:15:57.269059 (+ 157us) client_negotiation.cc:769] Sending connection context
0923 23:15:57.269560 (+ 501us) client_negotiation.cc:240] Negotiation successful
0923 23:15:57.269720 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":633,"thread_start_us":85,"threads_started":1}
I20260923 23:15:57.270392 4702 heartbeater.cc:347] Connected to a master server at 127.24.227.254:44363
I20260923 23:15:57.270490 4702 heartbeater.cc:464] Registering TS with master...
I20260923 23:15:57.270674 4702 heartbeater.cc:511] Master 127.24.227.254:44363 requested a full tablet report, sending...
I20260923 23:15:57.271016 4218 ts_manager.cc:194] Registered new tserver with Master: fe67dff8bc4e406187a56a2c48bb5ac7 (127.24.227.195:34811)
I20260923 23:15:57.271507 4218 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.227.195@KRBTEST.COM'} at 127.24.227.195:44463
I20260923 23:15:57.272346 25487 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 23:15:57.295953 4430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:15:57.280160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33678 (local address 127.24.227.254:44363)
0923 23:15:57.280207 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0923 23:15:57.280209 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:15:57.287963 (+ 7754us) server_negotiation.cc:407] Connection header received
0923 23:15:57.287997 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:15:57.287999 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:15:57.288011 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:15:57.288078 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:15:57.288874 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:57.290032 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:57.291082 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:57.291376 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:57.291451 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:57.292986 (+ 1535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:15:57.292988 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:15:57.292988 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:15:57.292996 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:15:57.294712 (+ 1716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:15:57.295112 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:15:57.295113 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:15:57.295113 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:15:57.295177 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:15:57.295390 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:15:57.295391 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:15:57.295391 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:15:57.295523 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:15:57.295620 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0923 23:15:57.295755 (+ 135us) server_negotiation.cc:299] Negotiation successful
0923 23:15:57.295833 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260923 23:15:57.297710 4218 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33678:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 23:15:57.300485 4218 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 23:15:57.308445 4716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:15:57.303009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.195:34811 (local address 127.0.0.1:43898)
0923 23:15:57.303451 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0923 23:15:57.303489 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0923 23:15:57.303547 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:15:57.304533 (+ 986us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:15:57.304536 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:15:57.304539 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:15:57.304608 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:15:57.304610 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:15:57.305953 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:15:57.305955 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:15:57.307238 (+ 1283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:15:57.307241 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:15:57.308198 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:15:57.308199 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:15:57.308274 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:57.308288 (+ 14us) client_negotiation.cc:769] Sending connection context
0923 23:15:57.308329 (+ 41us) client_negotiation.cc:240] Negotiation successful
0923 23:15:57.308375 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":70,"threads_started":1}
I20260923 23:15:57.309113 4719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:15:57.304125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43898 (local address 127.24.227.195:34811)
0923 23:15:57.304333 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0923 23:15:57.304336 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 23:15:57.304346 (+ 10us) server_negotiation.cc:407] Connection header received
0923 23:15:57.304384 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:15:57.304387 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:15:57.304420 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:15:57.304688 (+ 268us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:15:57.304760 (+ 72us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:57.305855 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:57.307358 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:57.308105 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:57.308908 (+ 803us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:57.308975 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0923 23:15:57.309001 (+ 26us) server_negotiation.cc:299] Negotiation successful
0923 23:15:57.309038 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":124,"threads_started":1}
I20260923 23:15:57.309957 4636 tablet_service.cc:1467] Processing CreateTablet for tablet 26f465772c0f4e9f9a879beefcf3a5b1 (DEFAULT_TABLE table=test-table [id=695f7078bd9b4f9990c9a74aec4241f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 23:15:57.310333 4636 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26f465772c0f4e9f9a879beefcf3a5b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:15:57.324798 4714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:15:57.303019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.194:35449 (local address 127.0.0.1:52860)
0923 23:15:57.304755 (+ 1736us) negotiation.cc:107] Waiting for socket to connect
0923 23:15:57.304766 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 23:15:57.304821 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:15:57.305080 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:15:57.305081 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:15:57.305083 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:15:57.305129 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:15:57.305130 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:15:57.307391 (+ 2261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:15:57.307392 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:15:57.323206 (+ 15814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:15:57.323208 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:15:57.324560 (+ 1352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:15:57.324561 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:15:57.324622 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:57.324626 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 23:15:57.324675 (+ 49us) client_negotiation.cc:240] Negotiation successful
0923 23:15:57.324728 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1719,"thread_start_us":87,"threads_started":1}
I20260923 23:15:57.326313 4717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:15:57.303344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52860 (local address 127.24.227.194:35449)
0923 23:15:57.304696 (+ 1352us) server_negotiation.cc:206] Beginning negotiation
0923 23:15:57.304701 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 23:15:57.304878 (+ 177us) server_negotiation.cc:407] Connection header received
0923 23:15:57.304923 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:15:57.304926 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:15:57.304942 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:15:57.305013 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:15:57.305207 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:57.307291 (+ 2084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:57.323333 (+ 16042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:57.324383 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:57.324473 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:57.324526 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0923 23:15:57.326190 (+ 1664us) server_negotiation.cc:299] Negotiation successful
0923 23:15:57.326234 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1298,"thread_start_us":95,"threads_started":1}
I20260923 23:15:57.326994 4501 tablet_service.cc:1467] Processing CreateTablet for tablet 26f465772c0f4e9f9a879beefcf3a5b1 (DEFAULT_TABLE table=test-table [id=695f7078bd9b4f9990c9a74aec4241f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 23:15:57.327355 4501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26f465772c0f4e9f9a879beefcf3a5b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:15:57.328059 4720 tablet_bootstrap.cc:492] T 26f465772c0f4e9f9a879beefcf3a5b1 P fe67dff8bc4e406187a56a2c48bb5ac7: Bootstrap starting.
I20260923 23:15:57.329043 4720 tablet_bootstrap.cc:654] T 26f465772c0f4e9f9a879beefcf3a5b1 P fe67dff8bc4e406187a56a2c48bb5ac7: Neither blocks nor log segments found. Creating new log.
I20260923 23:15:57.329347 4720 log.cc:824] T 26f465772c0f4e9f9a879beefcf3a5b1 P fe67dff8bc4e406187a56a2c48bb5ac7: Log is configured to *not* fsync() on all Append() calls
I20260923 23:15:57.330171 4720 tablet_bootstrap.cc:492] T 26f465772c0f4e9f9a879beefcf3a5b1 P fe67dff8bc4e406187a56a2c48bb5ac7: No bootstrap required, opened a new log
I20260923 23:15:57.330243 4720 ts_tablet_manager.cc:1397] T 26f465772c0f4e9f9a879beefcf3a5b1 P fe67dff8bc4e406187a56a2c48bb5ac7: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260923 23:15:57.330389 4722 tablet_bootstrap.cc:492] T 26f465772c0f4e9f9a879beefcf3a5b1 P 490732101ced4f2fa34f753d6706da1c: Bootstrap starting.
I20260923 23:15:57.331374 4722 tablet_bootstrap.cc:654] T 26f465772c0f4e9f9a879beefcf3a5b1 P 490732101ced4f2fa34f753d6706da1c: Neither blocks nor log segments found. Creating new log.
I20260923 23:15:57.331709 4718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:15:57.303938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38992 (local address 127.24.227.193:43451)
0923 23:15:57.304094 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0923 23:15:57.304099 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 23:15:57.306845 (+ 2746us) server_negotiation.cc:407] Connection header received
0923 23:15:57.309237 (+ 2392us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:15:57.309240 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:15:57.309260 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:15:57.309525 (+ 265us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:15:57.309778 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:57.310934 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:57.325035 (+ 14101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:57.325871 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:57.325925 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:57.325973 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0923 23:15:57.331550 (+ 5577us) server_negotiation.cc:299] Negotiation successful
0923 23:15:57.331609 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":79,"threads_started":1}
I20260923 23:15:57.331725 4722 log.cc:824] T 26f465772c0f4e9f9a879beefcf3a5b1 P 490732101ced4f2fa34f753d6706da1c: Log is configured to *not* fsync() on all Append() calls
I20260923 23:15:57.331979 4715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:15:57.303216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.193:43451 (local address 127.0.0.1:38992)
0923 23:15:57.306749 (+ 3533us) negotiation.cc:107] Waiting for socket to connect
0923 23:15:57.306760 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 23:15:57.309132 (+ 2372us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:15:57.309637 (+ 505us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:15:57.309638 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:15:57.309640 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:15:57.309681 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:15:57.309694 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:15:57.311034 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:15:57.311035 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:15:57.322984 (+ 11949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:15:57.322987 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:15:57.326043 (+ 3056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:15:57.326044 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:15:57.330736 (+ 4692us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:57.330743 (+ 7us) client_negotiation.cc:769] Sending connection context
0923 23:15:57.331826 (+ 1083us) client_negotiation.cc:240] Negotiation successful
0923 23:15:57.331898 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3516,"thread_start_us":69,"threads_started":1}
I20260923 23:15:57.332100 4720 raft_consensus.cc:348] T 26f465772c0f4e9f9a879beefcf3a5b1 P fe67dff8bc4e406187a56a2c48bb5ac7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe67dff8bc4e406187a56a2c48bb5ac7" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 34811 } } peers { permanent_uuid: "490732101ced4f2fa34f753d6706da1c" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 35449 } } peers { permanent_uuid: "1649bed544794bfa8321d452f7d518e5" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 43451 } }
I20260923 23:15:57.332261 4720 raft_consensus.cc:374] T 26f465772c0f4e9f9a879beefcf3a5b1 P fe67dff8bc4e406187a56a2c48bb5ac7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:15:57.332296 4720 raft_consensus.cc:729] T 26f465772c0f4e9f9a879beefcf3a5b1 P fe67dff8bc4e406187a56a2c48bb5ac7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe67dff8bc4e406187a56a2c48bb5ac7, State: Initialized, Role: FOLLOWER
I20260923 23:15:57.332394 4720 consensus_queue.cc:260] T 26f465772c0f4e9f9a879beefcf3a5b1 P fe67dff8bc4e406187a56a2c48bb5ac7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe67dff8bc4e406187a56a2c48bb5ac7" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 34811 } } peers { permanent_uuid: "490732101ced4f2fa34f753d6706da1c" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 35449 } } peers { permanent_uuid: "1649bed544794bfa8321d452f7d518e5" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 43451 } }
I20260923 23:15:57.332623 4720 ts_tablet_manager.cc:1428] T 26f465772c0f4e9f9a879beefcf3a5b1 P fe67dff8bc4e406187a56a2c48bb5ac7: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260923 23:15:57.332628 4358 tablet_service.cc:1467] Processing CreateTablet for tablet 26f465772c0f4e9f9a879beefcf3a5b1 (DEFAULT_TABLE table=test-table [id=695f7078bd9b4f9990c9a74aec4241f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 23:15:57.332749 4702 heartbeater.cc:503] Master 127.24.227.254:44363 was elected leader, sending a full tablet report...
I20260923 23:15:57.332969 4358 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26f465772c0f4e9f9a879beefcf3a5b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:15:57.333441 4722 tablet_bootstrap.cc:492] T 26f465772c0f4e9f9a879beefcf3a5b1 P 490732101ced4f2fa34f753d6706da1c: No bootstrap required, opened a new log
I20260923 23:15:57.333552 4722 ts_tablet_manager.cc:1397] T 26f465772c0f4e9f9a879beefcf3a5b1 P 490732101ced4f2fa34f753d6706da1c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260923 23:15:57.335829 4722 raft_consensus.cc:348] T 26f465772c0f4e9f9a879beefcf3a5b1 P 490732101ced4f2fa34f753d6706da1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe67dff8bc4e406187a56a2c48bb5ac7" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 34811 } } peers { permanent_uuid: "490732101ced4f2fa34f753d6706da1c" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 35449 } } peers { permanent_uuid: "1649bed544794bfa8321d452f7d518e5" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 43451 } }
I20260923 23:15:57.336006 4722 raft_consensus.cc:374] T 26f465772c0f4e9f9a879beefcf3a5b1 P 490732101ced4f2fa34f753d6706da1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:15:57.336050 4722 raft_consensus.cc:729] T 26f465772c0f4e9f9a879beefcf3a5b1 P 490732101ced4f2fa34f753d6706da1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 490732101ced4f2fa34f753d6706da1c, State: Initialized, Role: FOLLOWER
I20260923 23:15:57.336194 4722 consensus_queue.cc:260] T 26f465772c0f4e9f9a879beefcf3a5b1 P 490732101ced4f2fa34f753d6706da1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe67dff8bc4e406187a56a2c48bb5ac7" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 34811 } } peers { permanent_uuid: "490732101ced4f2fa34f753d6706da1c" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 35449 } } peers { permanent_uuid: "1649bed544794bfa8321d452f7d518e5" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 43451 } }
I20260923 23:15:57.336436 4722 ts_tablet_manager.cc:1428] T 26f465772c0f4e9f9a879beefcf3a5b1 P 490732101ced4f2fa34f753d6706da1c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 23:15:57.336549 4567 heartbeater.cc:503] Master 127.24.227.254:44363 was elected leader, sending a full tablet report...
I20260923 23:15:57.337442 4725 tablet_bootstrap.cc:492] T 26f465772c0f4e9f9a879beefcf3a5b1 P 1649bed544794bfa8321d452f7d518e5: Bootstrap starting.
I20260923 23:15:57.337976 4725 tablet_bootstrap.cc:654] T 26f465772c0f4e9f9a879beefcf3a5b1 P 1649bed544794bfa8321d452f7d518e5: Neither blocks nor log segments found. Creating new log.
I20260923 23:15:57.338256 4725 log.cc:824] T 26f465772c0f4e9f9a879beefcf3a5b1 P 1649bed544794bfa8321d452f7d518e5: Log is configured to *not* fsync() on all Append() calls
I20260923 23:15:57.341403 4725 tablet_bootstrap.cc:492] T 26f465772c0f4e9f9a879beefcf3a5b1 P 1649bed544794bfa8321d452f7d518e5: No bootstrap required, opened a new log
I20260923 23:15:57.341461 4725 ts_tablet_manager.cc:1397] T 26f465772c0f4e9f9a879beefcf3a5b1 P 1649bed544794bfa8321d452f7d518e5: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260923 23:15:57.344692 4725 raft_consensus.cc:348] T 26f465772c0f4e9f9a879beefcf3a5b1 P 1649bed544794bfa8321d452f7d518e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe67dff8bc4e406187a56a2c48bb5ac7" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 34811 } } peers { permanent_uuid: "490732101ced4f2fa34f753d6706da1c" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 35449 } } peers { permanent_uuid: "1649bed544794bfa8321d452f7d518e5" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 43451 } }
I20260923 23:15:57.344848 4725 raft_consensus.cc:374] T 26f465772c0f4e9f9a879beefcf3a5b1 P 1649bed544794bfa8321d452f7d518e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:15:57.344887 4725 raft_consensus.cc:729] T 26f465772c0f4e9f9a879beefcf3a5b1 P 1649bed544794bfa8321d452f7d518e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1649bed544794bfa8321d452f7d518e5, State: Initialized, Role: FOLLOWER
I20260923 23:15:57.344987 4725 consensus_queue.cc:260] T 26f465772c0f4e9f9a879beefcf3a5b1 P 1649bed544794bfa8321d452f7d518e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe67dff8bc4e406187a56a2c48bb5ac7" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 34811 } } peers { permanent_uuid: "490732101ced4f2fa34f753d6706da1c" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 35449 } } peers { permanent_uuid: "1649bed544794bfa8321d452f7d518e5" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 43451 } }
I20260923 23:15:57.345232 4725 ts_tablet_manager.cc:1428] T 26f465772c0f4e9f9a879beefcf3a5b1 P 1649bed544794bfa8321d452f7d518e5: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260923 23:15:57.345335 4428 heartbeater.cc:503] Master 127.24.227.254:44363 was elected leader, sending a full tablet report...
Sep 23 23:15:57 dist-test-slave-754n krb5kdc[3742](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205356, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.254@KRBTEST.COM for HTTP/127.24.227.212@KRBTEST.COM
W20260923 23:15:57.497867 4568 tablet.cc:2369] T 26f465772c0f4e9f9a879beefcf3a5b1 P 490732101ced4f2fa34f753d6706da1c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 23:15:57.509804 4703 tablet.cc:2369] T 26f465772c0f4e9f9a879beefcf3a5b1 P fe67dff8bc4e406187a56a2c48bb5ac7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 23:15:57.509995 4429 tablet.cc:2369] T 26f465772c0f4e9f9a879beefcf3a5b1 P 1649bed544794bfa8321d452f7d518e5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 23:15:57.526973 4724 raft_consensus.cc:482] T 26f465772c0f4e9f9a879beefcf3a5b1 P fe67dff8bc4e406187a56a2c48bb5ac7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 23:15:57.527117 4724 raft_consensus.cc:504] T 26f465772c0f4e9f9a879beefcf3a5b1 P fe67dff8bc4e406187a56a2c48bb5ac7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe67dff8bc4e406187a56a2c48bb5ac7" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 34811 } } peers { permanent_uuid: "490732101ced4f2fa34f753d6706da1c" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 35449 } } peers { permanent_uuid: "1649bed544794bfa8321d452f7d518e5" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 43451 } }
I20260923 23:15:57.527525 4724 leader_election.cc:288] T 26f465772c0f4e9f9a879beefcf3a5b1 P fe67dff8bc4e406187a56a2c48bb5ac7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 490732101ced4f2fa34f753d6706da1c (127.24.227.194:35449), 1649bed544794bfa8321d452f7d518e5 (127.24.227.193:43451)
I20260923 23:15:57.531459 4704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:15:57.527758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.194:35449 (local address 127.24.227.195:56267)
0923 23:15:57.527949 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0923 23:15:57.527964 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0923 23:15:57.528009 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:15:57.528183 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:15:57.528185 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:15:57.528188 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:15:57.528262 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:15:57.528264 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:15:57.529202 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:15:57.529203 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:15:57.530448 (+ 1245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:15:57.530450 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:15:57.531221 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:15:57.531222 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:15:57.531304 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:57.531308 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 23:15:57.531353 (+ 45us) client_negotiation.cc:240] Negotiation successful
0923 23:15:57.531389 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174}
I20260923 23:15:57.531750 4717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:15:57.527799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.195:56267 (local address 127.24.227.194:35449)
0923 23:15:57.527846 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0923 23:15:57.527849 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 23:15:57.528044 (+ 195us) server_negotiation.cc:407] Connection header received
0923 23:15:57.528075 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:15:57.528076 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:15:57.528086 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:15:57.528145 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:15:57.528364 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:57.529102 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:57.530548 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:57.531108 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:57.531182 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:57.531190 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0923 23:15:57.531649 (+ 459us) server_negotiation.cc:299] Negotiation successful
0923 23:15:57.531689 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260923 23:15:57.532088 4510 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26f465772c0f4e9f9a879beefcf3a5b1" candidate_uuid: "fe67dff8bc4e406187a56a2c48bb5ac7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "490732101ced4f2fa34f753d6706da1c" is_pre_election: true
I20260923 23:15:57.532244 4510 raft_consensus.cc:2454] T 26f465772c0f4e9f9a879beefcf3a5b1 P 490732101ced4f2fa34f753d6706da1c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fe67dff8bc4e406187a56a2c48bb5ac7 in term 0.
I20260923 23:15:57.533057 4590 leader_election.cc:302] T 26f465772c0f4e9f9a879beefcf3a5b1 P fe67dff8bc4e406187a56a2c48bb5ac7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 490732101ced4f2fa34f753d6706da1c, fe67dff8bc4e406187a56a2c48bb5ac7; no voters:
I20260923 23:15:57.533202 4724 raft_consensus.cc:2781] T 26f465772c0f4e9f9a879beefcf3a5b1 P fe67dff8bc4e406187a56a2c48bb5ac7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 23:15:57.533270 4724 raft_consensus.cc:482] T 26f465772c0f4e9f9a879beefcf3a5b1 P fe67dff8bc4e406187a56a2c48bb5ac7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 23:15:57.533291 4724 raft_consensus.cc:3037] T 26f465772c0f4e9f9a879beefcf3a5b1 P fe67dff8bc4e406187a56a2c48bb5ac7 [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:15:57.534104 4724 raft_consensus.cc:504] T 26f465772c0f4e9f9a879beefcf3a5b1 P fe67dff8bc4e406187a56a2c48bb5ac7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe67dff8bc4e406187a56a2c48bb5ac7" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 34811 } } peers { permanent_uuid: "490732101ced4f2fa34f753d6706da1c" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 35449 } } peers { permanent_uuid: "1649bed544794bfa8321d452f7d518e5" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 43451 } }
I20260923 23:15:57.534247 4724 leader_election.cc:288] T 26f465772c0f4e9f9a879beefcf3a5b1 P fe67dff8bc4e406187a56a2c48bb5ac7 [CANDIDATE]: Term 1 election: Requested vote from peers 490732101ced4f2fa34f753d6706da1c (127.24.227.194:35449), 1649bed544794bfa8321d452f7d518e5 (127.24.227.193:43451)
I20260923 23:15:57.534937 4510 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26f465772c0f4e9f9a879beefcf3a5b1" candidate_uuid: "fe67dff8bc4e406187a56a2c48bb5ac7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "490732101ced4f2fa34f753d6706da1c"
I20260923 23:15:57.535050 4510 raft_consensus.cc:3037] T 26f465772c0f4e9f9a879beefcf3a5b1 P 490732101ced4f2fa34f753d6706da1c [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:15:57.535082 4704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:15:57.528096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.193:43451 (local address 127.24.227.195:51013)
0923 23:15:57.531538 (+ 3442us) negotiation.cc:107] Waiting for socket to connect
0923 23:15:57.531548 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 23:15:57.531586 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:15:57.532114 (+ 528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:15:57.532115 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:15:57.532116 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:15:57.532149 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:15:57.532150 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:15:57.533164 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:15:57.533164 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:15:57.533888 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:15:57.533890 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:15:57.534868 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:15:57.534869 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:15:57.534927 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:57.534931 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 23:15:57.534983 (+ 52us) client_negotiation.cc:240] Negotiation successful
0923 23:15:57.535015 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3431,"thread_start_us":110,"threads_started":1}
I20260923 23:15:57.535411 4718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:15:57.527987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.195:51013 (local address 127.24.227.193:43451)
0923 23:15:57.528026 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0923 23:15:57.528028 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:15:57.531741 (+ 3713us) server_negotiation.cc:407] Connection header received
0923 23:15:57.531772 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:15:57.531773 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:15:57.531783 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:15:57.531839 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:15:57.532231 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:57.532863 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:57.533981 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:57.534515 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:57.534586 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:57.534593 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0923 23:15:57.535306 (+ 713us) server_negotiation.cc:299] Negotiation successful
0923 23:15:57.535347 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 23:15:57.535666 4382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26f465772c0f4e9f9a879beefcf3a5b1" candidate_uuid: "fe67dff8bc4e406187a56a2c48bb5ac7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1649bed544794bfa8321d452f7d518e5" is_pre_election: true
I20260923 23:15:57.535797 4510 raft_consensus.cc:2454] T 26f465772c0f4e9f9a879beefcf3a5b1 P 490732101ced4f2fa34f753d6706da1c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fe67dff8bc4e406187a56a2c48bb5ac7 in term 1.
I20260923 23:15:57.535823 4382 raft_consensus.cc:2454] T 26f465772c0f4e9f9a879beefcf3a5b1 P 1649bed544794bfa8321d452f7d518e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fe67dff8bc4e406187a56a2c48bb5ac7 in term 0.
I20260923 23:15:57.535979 4381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26f465772c0f4e9f9a879beefcf3a5b1" candidate_uuid: "fe67dff8bc4e406187a56a2c48bb5ac7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1649bed544794bfa8321d452f7d518e5"
I20260923 23:15:57.536048 4381 raft_consensus.cc:3037] T 26f465772c0f4e9f9a879beefcf3a5b1 P 1649bed544794bfa8321d452f7d518e5 [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:15:57.536893 4381 raft_consensus.cc:2454] T 26f465772c0f4e9f9a879beefcf3a5b1 P 1649bed544794bfa8321d452f7d518e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fe67dff8bc4e406187a56a2c48bb5ac7 in term 1.
I20260923 23:15:57.537145 4589 leader_election.cc:302] T 26f465772c0f4e9f9a879beefcf3a5b1 P fe67dff8bc4e406187a56a2c48bb5ac7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1649bed544794bfa8321d452f7d518e5, fe67dff8bc4e406187a56a2c48bb5ac7; no voters:
I20260923 23:15:57.537431 4724 raft_consensus.cc:2781] T 26f465772c0f4e9f9a879beefcf3a5b1 P fe67dff8bc4e406187a56a2c48bb5ac7 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 23:15:57.537583 4724 raft_consensus.cc:686] T 26f465772c0f4e9f9a879beefcf3a5b1 P fe67dff8bc4e406187a56a2c48bb5ac7 [term 1 LEADER]: Becoming Leader. State: Replica: fe67dff8bc4e406187a56a2c48bb5ac7, State: Running, Role: LEADER
I20260923 23:15:57.537686 4724 consensus_queue.cc:237] T 26f465772c0f4e9f9a879beefcf3a5b1 P fe67dff8bc4e406187a56a2c48bb5ac7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe67dff8bc4e406187a56a2c48bb5ac7" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 34811 } } peers { permanent_uuid: "490732101ced4f2fa34f753d6706da1c" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 35449 } } peers { permanent_uuid: "1649bed544794bfa8321d452f7d518e5" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 43451 } }
I20260923 23:15:57.538463 4217 catalog_manager.cc:5495] T 26f465772c0f4e9f9a879beefcf3a5b1 P fe67dff8bc4e406187a56a2c48bb5ac7 reported cstate change: term changed from 0 to 1, leader changed from <none> to fe67dff8bc4e406187a56a2c48bb5ac7 (127.24.227.195). New cstate: current_term: 1 leader_uuid: "fe67dff8bc4e406187a56a2c48bb5ac7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe67dff8bc4e406187a56a2c48bb5ac7" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 34811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "490732101ced4f2fa34f753d6706da1c" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 35449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1649bed544794bfa8321d452f7d518e5" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 43451 } health_report { overall_health: UNKNOWN } } }
I20260923 23:15:57.596230 4719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:15:57.593026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43904 (local address 127.24.227.195:34811)
0923 23:15:57.593086 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0923 23:15:57.593088 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:15:57.593129 (+ 41us) server_negotiation.cc:407] Connection header received
0923 23:15:57.593213 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:15:57.593214 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:15:57.593236 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:15:57.593314 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 23:15:57.593480 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:57.594187 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:57.595244 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:15:57.595561 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:15:57.595630 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:15:57.595829 (+ 199us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 23:15:57.595973 (+ 144us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 23:15:57.596054 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0923 23:15:57.596108 (+ 54us) server_negotiation.cc:299] Negotiation successful
0923 23:15:57.596127 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260923 23:15:57.598279 4381 raft_consensus.cc:1261] T 26f465772c0f4e9f9a879beefcf3a5b1 P 1649bed544794bfa8321d452f7d518e5 [term 1 FOLLOWER]: Refusing update from remote peer fe67dff8bc4e406187a56a2c48bb5ac7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 23:15:57.598309 4510 raft_consensus.cc:1261] T 26f465772c0f4e9f9a879beefcf3a5b1 P 490732101ced4f2fa34f753d6706da1c [term 1 FOLLOWER]: Refusing update from remote peer fe67dff8bc4e406187a56a2c48bb5ac7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 23:15:57.598695 4730 consensus_queue.cc:1035] T 26f465772c0f4e9f9a879beefcf3a5b1 P fe67dff8bc4e406187a56a2c48bb5ac7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1649bed544794bfa8321d452f7d518e5" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 43451 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 23:15:57.598794 4724 consensus_queue.cc:1035] T 26f465772c0f4e9f9a879beefcf3a5b1 P fe67dff8bc4e406187a56a2c48bb5ac7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "490732101ced4f2fa34f753d6706da1c" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 35449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 23:15:57.606834 4735 mvcc.cc:205] Tried to move back new op lower bound from 7332681144719527936 to 7332681144474738688. Current Snapshot: MvccSnapshot[applied={T|T < 7332681144719527936}]
I20260923 23:15:57.606932 4733 mvcc.cc:205] Tried to move back new op lower bound from 7332681144719527936 to 7332681144474738688. Current Snapshot: MvccSnapshot[applied={T|T < 7332681144719527936}]
I20260923 23:15:57.607035 4734 mvcc.cc:205] Tried to move back new op lower bound from 7332681144719527936 to 7332681144474738688. Current Snapshot: MvccSnapshot[applied={T|T < 7332681144719527936}]
I20260923 23:15:57.609334 4218 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33678:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 23:15:57.609427 4218 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33678:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 23:15:57.610116 4218 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bb36251ef7cb4a3eb3f41d0813b7de1f: Sending DeleteTablet for 3 replicas of tablet 26f465772c0f4e9f9a879beefcf3a5b1
I20260923 23:15:57.610425 4636 tablet_service.cc:1514] Processing DeleteTablet for tablet 26f465772c0f4e9f9a879beefcf3a5b1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 23:15:57 UTC) from {username='kudu', principal='kudu/127.24.227.254@KRBTEST.COM'} at 127.0.0.1:43898
I20260923 23:15:57.610462 4358 tablet_service.cc:1514] Processing DeleteTablet for tablet 26f465772c0f4e9f9a879beefcf3a5b1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 23:15:57 UTC) from {username='kudu', principal='kudu/127.24.227.254@KRBTEST.COM'} at 127.0.0.1:38992
I20260923 23:15:57.610574 4501 tablet_service.cc:1514] Processing DeleteTablet for tablet 26f465772c0f4e9f9a879beefcf3a5b1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 23:15:57 UTC) from {username='kudu', principal='kudu/127.24.227.254@KRBTEST.COM'} at 127.0.0.1:52860
I20260923 23:15:57.611038 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 4299
I20260923 23:15:57.615983 4744 tablet_replica.cc:330] T 26f465772c0f4e9f9a879beefcf3a5b1 P fe67dff8bc4e406187a56a2c48bb5ac7: stopping tablet replica
I20260923 23:15:57.619736 4744 raft_consensus.cc:2229] T 26f465772c0f4e9f9a879beefcf3a5b1 P fe67dff8bc4e406187a56a2c48bb5ac7 [term 1 LEADER]: Raft consensus shutting down.
I20260923 23:15:57.619949 4744 raft_consensus.cc:2258] T 26f465772c0f4e9f9a879beefcf3a5b1 P fe67dff8bc4e406187a56a2c48bb5ac7 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260923 23:15:57.620209 4202 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.227.193:43451 (error 108)
I20260923 23:15:57.620307 4745 tablet_replica.cc:330] T 26f465772c0f4e9f9a879beefcf3a5b1 P 490732101ced4f2fa34f753d6706da1c: stopping tablet replica
I20260923 23:15:57.620399 4745 raft_consensus.cc:2229] T 26f465772c0f4e9f9a879beefcf3a5b1 P 490732101ced4f2fa34f753d6706da1c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 23:15:57.620467 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 4436
I20260923 23:15:57.621443 4744 ts_tablet_manager.cc:1905] T 26f465772c0f4e9f9a879beefcf3a5b1 P fe67dff8bc4e406187a56a2c48bb5ac7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 23:15:57.623368 4744 ts_tablet_manager.cc:1918] T 26f465772c0f4e9f9a879beefcf3a5b1 P fe67dff8bc4e406187a56a2c48bb5ac7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 23:15:57.623445 4744 log.cc:1196] T 26f465772c0f4e9f9a879beefcf3a5b1 P fe67dff8bc4e406187a56a2c48bb5ac7: Deleting WAL directory at /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790205259943518-25487-0/minicluster-data/ts-2/wal/wals/26f465772c0f4e9f9a879beefcf3a5b1
I20260923 23:15:57.623719 4744 ts_tablet_manager.cc:1939] T 26f465772c0f4e9f9a879beefcf3a5b1 P fe67dff8bc4e406187a56a2c48bb5ac7: Deleting consensus metadata
I20260923 23:15:57.624106 4205 catalog_manager.cc:4841] TS fe67dff8bc4e406187a56a2c48bb5ac7 (127.24.227.195:34811): tablet 26f465772c0f4e9f9a879beefcf3a5b1 (table test-table [id=695f7078bd9b4f9990c9a74aec4241f1]) successfully deleted
I20260923 23:15:57.627481 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 4572
I20260923 23:15:57.628240 4715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:15:57.627636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.193:43451 (local address 127.0.0.1:38998)
0923 23:15:57.628162 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0923 23:15:57.628208 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.193:43451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":505}
W20260923 23:15:57.629065 4202 catalog_manager.cc:4584] TS 1649bed544794bfa8321d452f7d518e5 (127.24.227.193:43451): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 26f465772c0f4e9f9a879beefcf3a5b1: Network error: Client connection negotiation failed: client connection to 127.24.227.193:43451: connect: Connection refused (error 111)
I20260923 23:15:57.629452 4715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:15:57.629356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.194:35449 (local address 127.0.0.1:52876)
0923 23:15:57.629396 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0923 23:15:57.629419 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.194:35449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260923 23:15:57.632084 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 4186
I20260923 23:15:57.637552 25487 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260923 23:15:57.971125 25487 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260923 23:15:57.971251 25487 mini_ranger.cc:66] Stopping Ranger...
I20260923 23:15:58.315137 25487 mini_ranger.cc:68] Stopped Ranger
2026-09-23 23:15:58.315 UTC [3772] LOG: received smart shutdown request
2026-09-23 23:15:58.318 UTC [3772] LOG: background worker "logical replication launcher" (PID 3781) exited with exit code 1
2026-09-23 23:15:58.318 UTC [3776] LOG: shutting down
2026-09-23 23:15:58.458 UTC [3772] LOG: database system is shut down
2026-09-23T23:15:58Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (44349 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 23:15:58 dist-test-slave-754n krb5kdc[4755](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 23:15:58 dist-test-slave-754n krb5kdc[4755](info): set up 2 sockets
Sep 23 23:15:58 dist-test-slave-754n krb5kdc[4755](info): commencing operation
krb5kdc: starting...
W20260923 23:15:59.591747 25487 mini_kdc.cc:121] Time spent starting KDC: real 1.004s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/krb5kdc/test-user.keytab.
Sep 23 23:15:59 dist-test-slave-754n krb5kdc[4755](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205359, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T23:15:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T23:15:59Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.24.227.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.24.227.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.24.227.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/krb5kdc/rangeradmin_127.24.227.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.24.227.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/krb5kdc/rangeradmin_127.24.227.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.24.227.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.24.227.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.24.227.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/krb5kdc/rangerlookup_127.24.227.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.24.227.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/krb5kdc/rangerlookup_127.24.227.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.24.227.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.24.227.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.24.227.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/krb5kdc/HTTP_127.24.227.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.24.227.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/krb5kdc/HTTP_127.24.227.212@KRBTEST.COM.keytab.
I20260923 23:15:59.740583 25487 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskyHY_dZ/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/postgres -l logfile start
2026-09-23 23:16:02.016 UTC [4785] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-23 23:16:02.016 UTC [4785] LOG: listening on IPv4 address "127.24.227.212", port 33879
2026-09-23 23:16:02.019 UTC [4785] LOG: listening on Unix socket "/tmp/.s.PGSQL.33879"
2026-09-23 23:16:02.039 UTC [4788] LOG: database system was shut down at 2026-09-23 23:16:00 UTC
2026-09-23 23:16:02.042 UTC [4785] LOG: database system is ready to accept connections
I20260923 23:16:02.983652 25487 mini_postgres.cc:96] Postgres bound to 33879
2026-09-23 23:16:02.989 UTC [4796] FATAL: database "slave" does not exist
127.24.227.212:33879 - accepting connections
I20260923 23:16:02.990121 25487 mini_ranger.cc:161] Starting Ranger...
I20260923 23:16:03.003646 25487 mini_ranger.cc:84] Created miniranger Postgres user
I20260923 23:16:03.418134 25487 mini_ranger.cc:87] Created ranger Postgres database
I20260923 23:16:03.418232 25487 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ranger-admin
2026-09-23 23:16:03,681 [I] DB FLAVOR :POSTGRES
2026-09-23 23:16:03,681 [I] --------- Verifying Ranger DB connection ---------
2026-09-23 23:16:03,682 [I] Checking connection..
2026-09-23 23:16:03,682 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyHY_dZ/build/release/bin/postgresql.jar:/tmp/dist-test-taskyHY_dZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.227.212:33879/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-23 23:16:03,919 [I] Checking connection passed.
2026-09-23 23:16:03,919 [I] --------- Verifying version history table ---------
2026-09-23 23:16:03,919 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyHY_dZ/build/release/bin/postgresql.jar:/tmp/dist-test-taskyHY_dZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.227.212:33879/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-23 23:16:04,156 [I] Table x_db_version_h does not exist in database ranger
2026-09-23 23:16:04,157 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyHY_dZ/build/release/bin/postgresql.jar:/tmp/dist-test-taskyHY_dZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.227.212:33879/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-23 23:16:04,378 [I] Table x_db_version_h does not exist in database ranger
2026-09-23 23:16:04,378 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-23 23:16:04,378 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyHY_dZ/build/release/bin/postgresql.jar:/tmp/dist-test-taskyHY_dZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.227.212:33879/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskyHY_dZ/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-23 23:16:04.612 UTC [4860] WARNING: there is no transaction in progress
2026-09-23 23:16:04,617 [I] create_dbversion_catalog.sql file imported successfully
2026-09-23 23:16:04,618 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyHY_dZ/build/release/bin/postgresql.jar:/tmp/dist-test-taskyHY_dZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.227.212:33879/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-23 23:16:04,864 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-23 23:16:04,865 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-23 23:16:04,865 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyHY_dZ/build/release/bin/postgresql.jar:/tmp/dist-test-taskyHY_dZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.227.212:33879/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-23 23:16:05,093 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyHY_dZ/build/release/bin/postgresql.jar:/tmp/dist-test-taskyHY_dZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.227.212:33879/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-23 23:16:05,323 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyHY_dZ/build/release/bin/postgresql.jar:/tmp/dist-test-taskyHY_dZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.227.212:33879/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-754n.c.gcp-upstream.internal','N') ;"
2026-09-23 23:16:05,569 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyHY_dZ/build/release/bin/postgresql.jar:/tmp/dist-test-taskyHY_dZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.227.212:33879/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-23 23:16:05,819 [I] Table x_portal_user does not exist in database ranger
2026-09-23 23:16:05,819 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyHY_dZ/build/release/bin/postgresql.jar:/tmp/dist-test-taskyHY_dZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.227.212:33879/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-23 23:16:06,069 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-23 23:16:06,069 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyHY_dZ/build/release/bin/postgresql.jar:/tmp/dist-test-taskyHY_dZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.227.212:33879/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-23 23:16:06,303 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyHY_dZ/build/release/bin/postgresql.jar:/tmp/dist-test-taskyHY_dZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.227.212:33879/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-23 23:16:06,518 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-23 23:16:06,518 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyHY_dZ/build/release/bin/postgresql.jar:/tmp/dist-test-taskyHY_dZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.227.212:33879/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskyHY_dZ/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-23 23:16:07.423 UTC [4968] WARNING: there is no transaction in progress
2026-09-23 23:16:07.439 UTC [4968] WARNING: there is no transaction in progress
2026-09-23 23:16:07.455 UTC [4968] WARNING: there is no transaction in progress
2026-09-23 23:16:07.469 UTC [4968] WARNING: there is no transaction in progress
2026-09-23 23:16:07.485 UTC [4968] WARNING: there is no transaction in progress
2026-09-23 23:16:07.501 UTC [4968] WARNING: there is no transaction in progress
2026-09-23 23:16:07.553 UTC [4968] WARNING: there is no transaction in progress
2026-09-23 23:16:07.560 UTC [4968] WARNING: there is no transaction in progress
2026-09-23 23:16:07.574 UTC [4968] WARNING: there is no transaction in progress
2026-09-23 23:16:07.584 UTC [4968] WARNING: there is no transaction in progress
2026-09-23 23:16:07.594 UTC [4968] WARNING: there is no transaction in progress
2026-09-23 23:16:07.608 UTC [4968] WARNING: there is no transaction in progress
2026-09-23 23:16:07.617 UTC [4968] WARNING: there is no transaction in progress
2026-09-23 23:16:07.623 UTC [4968] WARNING: there is no transaction in progress
2026-09-23 23:16:07.630 UTC [4968] WARNING: there is no transaction in progress
2026-09-23 23:16:07.636 UTC [4968] WARNING: there is no transaction in progress
2026-09-23 23:16:08,281 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-23 23:16:08,281 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyHY_dZ/build/release/bin/postgresql.jar:/tmp/dist-test-taskyHY_dZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.227.212:33879/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-23 23:16:08,526 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-23 23:16:08,526 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyHY_dZ/build/release/bin/postgresql.jar:/tmp/dist-test-taskyHY_dZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.227.212:33879/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-23 23:16:08,747 [I] Table x_portal_user already exists in database 'ranger'
2026-09-23 23:16:08,747 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyHY_dZ/build/release/bin/postgresql.jar:/tmp/dist-test-taskyHY_dZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.227.212:33879/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-23 23:16:08,976 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-23 23:16:08,977 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyHY_dZ/build/release/bin/postgresql.jar:/tmp/dist-test-taskyHY_dZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.227.212:33879/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-23 23:16:09,198 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyHY_dZ/build/release/bin/postgresql.jar:/tmp/dist-test-taskyHY_dZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.227.212:33879/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-23 23:16:09,433 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyHY_dZ/build/release/bin/postgresql.jar:/tmp/dist-test-taskyHY_dZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.227.212:33879/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-754n.c.gcp-upstream.internal';"
2026-09-23 23:16:09,645 [I] CORE_DB_SCHEMA import status has been updated
2026-09-23 23:16:09,645 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyHY_dZ/build/release/bin/postgresql.jar:/tmp/dist-test-taskyHY_dZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.227.212:33879/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-23 23:16:09,890 [I] DB_PATCHES have already been applied
I20260923 23:16:09.895671 25487 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ranger-admin:/tmp/dist-test-taskyHY_dZ/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskyHY_dZ/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskyHY_dZ/build/release/bin/postgresql.jar:/tmp/dist-test-taskyHY_dZ/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260923 23:16:09.895728 25487 mini_ranger.cc:193] Using host: 127.24.227.212
I20260923 23:16:09.898763 25487 mini_ranger.cc:227] Ranger admin URL: http://127.24.227.212:58775
Sep 23, 2026 11:16:10 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 23, 2026 11:16:10 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 23, 2026 11:16:10 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ranger-admin/ews/webapp
Sep 23, 2026 11:16:10 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 23, 2026 11:16:10 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 23, 2026 11:16:10 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ranger-admin/ews/webapp].
Sep 23, 2026 11:16:10 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260923 23:16:10.998903 25487 mini_ranger.cc:160] Time spent starting Ranger: real 8.009s user 0.000s sys 0.008s
Sep 23, 2026 11:16:30 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 23, 2026 11:16:30 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Sep 23, 2026 11:16:30 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 23, 2026 11:16:30 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 23, 2026 11:16:30 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 23, 2026 11:16:30 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 23, 2026 11:16:30 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 23, 2026 11:16:30 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 23, 2026 11:16:30 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 23, 2026 11:16:30 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 23, 2026 11:16:30 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 23, 2026 11:16:30 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 23, 2026 11:16:30 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 23, 2026 11:16:30 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 23, 2026 11:16:30 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 23, 2026 11:16:30 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 23, 2026 11:16:30 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 23, 2026 11:16:31 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260923 23:16:31.697713 25487 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.24.227.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.24.227.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.24.227.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/krb5kdc/rangerkms_127.24.227.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.24.227.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/krb5kdc/rangerkms_127.24.227.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.24.227.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.24.227.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.24.227.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/krb5kdc/HTTP_127.24.227.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.24.227.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/krb5kdc/HTTP_127.24.227.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 23 23:16:31 dist-test-slave-754n krb5kdc[4755](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205391, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260923 23:16:31.775561 25487 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260923 23:16:31.784631 25487 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260923 23:16:32.562278 25487 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260923 23:16:32.562369 25487 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ranger-kms
I20260923 23:16:32.562389 25487 mini_ranger_kms.cc:227] Using postgres at 127.24.227.212:33879
2026-09-23 23:16:32,729 [I] DB FLAVOR :POSTGRES
2026-09-23 23:16:32,730 [I] --------- Verifying Ranger DB connection ---------
2026-09-23 23:16:32,730 [I] Checking connection
2026-09-23 23:16:32,730 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyHY_dZ/build/release/bin/postgresql.jar:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.227.212:33879/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-23 23:16:32,952 [I] connection success
2026-09-23 23:16:32,952 [I] --------- Verifying Ranger DB tables ---------
2026-09-23 23:16:32,952 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-23 23:16:32,952 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyHY_dZ/build/release/bin/postgresql.jar:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.227.212:33879/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-23 23:16:33,203 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-23 23:16:33,203 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-23 23:16:33,203 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-23 23:16:33,204 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskyHY_dZ/build/release/bin/postgresql.jar:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.227.212:33879/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-23 23:16:33,470 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260923 23:16:33.911592 25487 mini_ranger_kms.cc:310] Created kms service
I20260923 23:16:34.122349 25487 mini_ranger_kms.cc:326] Created kudu user
I20260923 23:16:34.162573 25487 mini_ranger_kms.cc:343] Created rangerkms user
Sep 23 23:16:34 dist-test-slave-754n krb5kdc[4755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205391, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.24.227.212@KRBTEST.COM
I20260923 23:16:34.538458 25487 mini_ranger_kms.cc:384] Added ranger policy
I20260923 23:16:34.538558 25487 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ranger-kms:/tmp/dist-test-taskyHY_dZ/build/release/bin/postgresql.jar:/tmp/dist-test-taskyHY_dZ/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskyHY_dZ/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260923 23:16:34.538575 25487 mini_ranger_kms.cc:240] Using host: 127.24.227.212
I20260923 23:16:34.541522 25487 mini_ranger_kms.cc:276] Ranger KMS PID: 5148
I20260923 23:16:34.541616 25487 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.24.227.212:35105
Sep 23, 2026 11:16:35 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 23, 2026 11:16:35 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 23, 2026 11:16:35 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 23, 2026 11:16:35 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 23, 2026 11:16:35 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ranger-kms/ews/webapp].
Sep 23, 2026 11:16:35 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Sep 23 23:16:36 dist-test-slave-754n krb5kdc[4755](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790205396, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.24.227.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:16:36.748088 25487 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 4.973s user 0.006s sys 0.144s
I20260923 23:16:36.748175 25487 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260923 23:16:36.748219 25487 mini_ranger_kms.cc:401] 127.24.227.212:35105/kms/v1/keys
Sep 23 23:16:36 dist-test-slave-754n krb5kdc[4755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205396, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.24.227.212@KRBTEST.COM for HTTP/127.24.227.212@KRBTEST.COM
Sep 23 23:16:37 dist-test-slave-754n krb5kdc[4755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205396, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.24.227.212@KRBTEST.COM for HTTP/127.24.227.212@KRBTEST.COM
Sep 23 23:16:39 dist-test-slave-754n krb5kdc[4755](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205399, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.24.227.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.227.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.227.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab.
Entry for principal kudu/127.24.227.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab.
I20260923 23:16:39.538723 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.254:33391
--webserver_interface=127.24.227.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab
--principal=kudu/127.24.227.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:41501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.227.254:33391
--ranger_config_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.227.212:35105/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:16:39.622888 5199 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:16:39.623078 5199 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:16:39.623098 5199 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:16:39.624287 5199 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 23:16:39.624326 5199 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:16:39.624341 5199 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:16:39.624353 5199 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 23:16:39.624365 5199 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 23:16:39.625989 5199 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:41501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/master-0/wal
--ranger_kms_url=127.24.227.212:35105/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.24.227.254:33391
--ranger_config_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab
--principal=kudu/127.24.227.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.254:33391
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.227.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5199
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:16:39.626214 5199 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:16:39.626453 5199 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:16:39.631285 5207 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:16:39.631323 5205 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:16:39.631379 5199 server_base.cc:1034] running on GCE node
W20260923 23:16:39.631420 5204 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:16:39.631862 5199 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:16:39.632210 5199 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:16:39.633380 5199 hybrid_clock.cc:648] HybridClock initialized: now 1790205399633363 us; error 32 us; skew 500 ppm
Sep 23 23:16:39 dist-test-slave-754n krb5kdc[4755](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205399, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:16:39.636336 5199 init.cc:377] Logged in from keytab as kudu/127.24.227.254@KRBTEST.COM (short username kudu)
I20260923 23:16:39.636767 5199 webserver.cc:460] Webserver started at http://127.24.227.254:42859/ using document root <none> and password file <none>
I20260923 23:16:39.636992 5199 fs_manager.cc:359] Metadata directory not provided
I20260923 23:16:39.637049 5199 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:16:39.637200 5199 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 23 23:16:39 dist-test-slave-754n krb5kdc[4755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205399, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.254@KRBTEST.COM for HTTP/127.24.227.212@KRBTEST.COM
I20260923 23:16:39.684223 5199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/master-0/data/instance:
uuid: "f2373ee6b47c4e8d9da1739e78341854"
format_stamp: "Formatted at 2026-09-23 23:16:39 on dist-test-slave-754n"
server_key: "0fc2e01f27d3593af043bafe76c7888f"
server_key_iv: "0ccc940fc251408dc74b4d9bb8386459"
server_key_version: "kuduclusterkey@0"
I20260923 23:16:39.684653 5199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/master-0/wal/instance:
uuid: "f2373ee6b47c4e8d9da1739e78341854"
format_stamp: "Formatted at 2026-09-23 23:16:39 on dist-test-slave-754n"
server_key: "0fc2e01f27d3593af043bafe76c7888f"
server_key_iv: "0ccc940fc251408dc74b4d9bb8386459"
server_key_version: "kuduclusterkey@0"
I20260923 23:16:39.686141 5199 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 23:16:39.698019 5214 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:16:39.700030 5199 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260923 23:16:39.700160 5199 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/master-0/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/master-0/wal
uuid: "f2373ee6b47c4e8d9da1739e78341854"
format_stamp: "Formatted at 2026-09-23 23:16:39 on dist-test-slave-754n"
server_key: "0fc2e01f27d3593af043bafe76c7888f"
server_key_iv: "0ccc940fc251408dc74b4d9bb8386459"
server_key_version: "kuduclusterkey@0"
I20260923 23:16:39.700263 5199 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:16:39.733177 5199 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:16:39.735292 5199 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:16:39.735437 5199 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:16:39.740226 5199 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.254:33391
I20260923 23:16:39.740317 5266 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.254:33391 every 8 connection(s)
I20260923 23:16:39.740650 5199 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
I20260923 23:16:39.741370 5267 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:16:39.743744 5267 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2373ee6b47c4e8d9da1739e78341854: Bootstrap starting.
I20260923 23:16:39.744376 5267 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f2373ee6b47c4e8d9da1739e78341854: Neither blocks nor log segments found. Creating new log.
I20260923 23:16:39.744710 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 5199
I20260923 23:16:39.744760 5267 log.cc:824] T 00000000000000000000000000000000 P f2373ee6b47c4e8d9da1739e78341854: Log is configured to *not* fsync() on all Append() calls
I20260923 23:16:39.744829 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/master-0/wal/instance
I20260923 23:16:39.745041 25487 external_mini_cluster.cc:1428] Setting key 25e8ca350df97310da6990d45ceda2a5
I20260923 23:16:39.745680 5267 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2373ee6b47c4e8d9da1739e78341854: No bootstrap required, opened a new log
I20260923 23:16:39.748252 5267 raft_consensus.cc:348] T 00000000000000000000000000000000 P f2373ee6b47c4e8d9da1739e78341854 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2373ee6b47c4e8d9da1739e78341854" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 33391 } }
I20260923 23:16:39.748433 5267 raft_consensus.cc:374] T 00000000000000000000000000000000 P f2373ee6b47c4e8d9da1739e78341854 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:16:39.748467 5267 raft_consensus.cc:729] T 00000000000000000000000000000000 P f2373ee6b47c4e8d9da1739e78341854 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2373ee6b47c4e8d9da1739e78341854, State: Initialized, Role: FOLLOWER
I20260923 23:16:39.748620 5267 consensus_queue.cc:260] T 00000000000000000000000000000000 P f2373ee6b47c4e8d9da1739e78341854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2373ee6b47c4e8d9da1739e78341854" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 33391 } }
I20260923 23:16:39.748690 5267 raft_consensus.cc:388] T 00000000000000000000000000000000 P f2373ee6b47c4e8d9da1739e78341854 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 23:16:39.748718 5267 raft_consensus.cc:482] T 00000000000000000000000000000000 P f2373ee6b47c4e8d9da1739e78341854 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 23:16:39.748780 5267 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f2373ee6b47c4e8d9da1739e78341854 [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:16:39.749410 5267 raft_consensus.cc:504] T 00000000000000000000000000000000 P f2373ee6b47c4e8d9da1739e78341854 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2373ee6b47c4e8d9da1739e78341854" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 33391 } }
I20260923 23:16:39.749532 5267 leader_election.cc:302] T 00000000000000000000000000000000 P f2373ee6b47c4e8d9da1739e78341854 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f2373ee6b47c4e8d9da1739e78341854; no voters:
I20260923 23:16:39.749691 5267 leader_election.cc:288] T 00000000000000000000000000000000 P f2373ee6b47c4e8d9da1739e78341854 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 23:16:39.749740 5272 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f2373ee6b47c4e8d9da1739e78341854 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 23:16:39.749944 5272 raft_consensus.cc:686] T 00000000000000000000000000000000 P f2373ee6b47c4e8d9da1739e78341854 [term 1 LEADER]: Becoming Leader. State: Replica: f2373ee6b47c4e8d9da1739e78341854, State: Running, Role: LEADER
I20260923 23:16:39.750039 5267 sys_catalog.cc:564] T 00000000000000000000000000000000 P f2373ee6b47c4e8d9da1739e78341854 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 23:16:39.750075 5272 consensus_queue.cc:237] T 00000000000000000000000000000000 P f2373ee6b47c4e8d9da1739e78341854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2373ee6b47c4e8d9da1739e78341854" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 33391 } }
I20260923 23:16:39.751725 5272 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2373ee6b47c4e8d9da1739e78341854 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f2373ee6b47c4e8d9da1739e78341854. Latest consensus state: current_term: 1 leader_uuid: "f2373ee6b47c4e8d9da1739e78341854" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2373ee6b47c4e8d9da1739e78341854" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 33391 } } }
I20260923 23:16:39.751929 5272 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2373ee6b47c4e8d9da1739e78341854 [sys.catalog]: This master's current role is: LEADER
I20260923 23:16:39.752066 5274 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2373ee6b47c4e8d9da1739e78341854 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f2373ee6b47c4e8d9da1739e78341854" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2373ee6b47c4e8d9da1739e78341854" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 33391 } } }
I20260923 23:16:39.752137 5274 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2373ee6b47c4e8d9da1739e78341854 [sys.catalog]: This master's current role is: LEADER
I20260923 23:16:39.752202 5286 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 23:16:39.752481 5267 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260923 23:16:39.752698 5286 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 23 23:16:39 dist-test-slave-754n krb5kdc[4755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205399, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
I20260923 23:16:39.757846 5271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:39.745554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47202 (local address 127.24.227.254:33391)
0923 23:16:39.745852 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0923 23:16:39.745861 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 23:16:39.745891 (+ 30us) server_negotiation.cc:407] Connection header received
0923 23:16:39.746886 (+ 995us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:16:39.746895 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:16:39.746957 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:16:39.747104 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:16:39.747854 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:39.749253 (+ 1399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:39.750320 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:39.751009 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:39.751130 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:16:39.754039 (+ 2909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:16:39.754043 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:16:39.754043 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:16:39.754055 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:16:39.755947 (+ 1892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:16:39.756361 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:16:39.756364 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:16:39.756364 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:16:39.756452 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:16:39.756640 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:16:39.756641 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:16:39.756641 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:16:39.756979 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:16:39.757114 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0923 23:16:39.757522 (+ 408us) server_negotiation.cc:299] Negotiation successful
0923 23:16:39.757633 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":248,"thread_start_us":75,"threads_started":1}
I20260923 23:16:39.762981 5286 catalog_manager.cc:1347] Generated new cluster ID: 17a85784336748afba6d6d4ba6627809
I20260923 23:16:39.763048 5286 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 23:16:39.779274 5286 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 23:16:39.780121 5286 catalog_manager.cc:1504] Loading token signing keys...
I20260923 23:16:39.791714 5286 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f2373ee6b47c4e8d9da1739e78341854: Generated new TSK 0
I20260923 23:16:39.792002 5286 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.227.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.227.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.227.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.193.keytab.
Entry for principal kudu/127.24.227.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.193.keytab.
I20260923 23:16:40.450728 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.227.212:35105/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.227.193:0
--local_ip_for_outbound_sockets=127.24.227.193
--webserver_interface=127.24.227.193
--webserver_port=0
--tserver_master_addrs=127.24.227.254:33391
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.193.keytab
--principal=kudu/127.24.227.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:41501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:16:40.563578 5312 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:16:40.563758 5312 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:16:40.563779 5312 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:16:40.564939 5312 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:16:40.564981 5312 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:16:40.565007 5312 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.227.193
I20260923 23:16:40.566712 5312 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:41501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.24.227.212:35105/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.193.keytab
--principal=kudu/127.24.227.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.227.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.227.193
--webserver_port=0
--tserver_master_addrs=127.24.227.254:33391
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5312
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.227.193
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:16:40.566937 5312 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:16:40.567173 5312 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:16:40.572069 5318 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:16:40.572127 5312 server_base.cc:1034] running on GCE node
W20260923 23:16:40.572052 5317 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:16:40.572252 5320 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:16:40.572501 5312 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:16:40.572757 5312 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:16:40.573915 5312 hybrid_clock.cc:648] HybridClock initialized: now 1790205400573905 us; error 27 us; skew 500 ppm
Sep 23 23:16:40 dist-test-slave-754n krb5kdc[4755](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205400, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:16:40.576747 5312 init.cc:377] Logged in from keytab as kudu/127.24.227.193@KRBTEST.COM (short username kudu)
I20260923 23:16:40.577265 5312 webserver.cc:460] Webserver started at http://127.24.227.193:44555/ using document root <none> and password file <none>
I20260923 23:16:40.577533 5312 fs_manager.cc:359] Metadata directory not provided
I20260923 23:16:40.577597 5312 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:16:40.577723 5312 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 23 23:16:40 dist-test-slave-754n krb5kdc[4755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205400, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.193@KRBTEST.COM for HTTP/127.24.227.212@KRBTEST.COM
I20260923 23:16:40.591096 5312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ts-0/data/instance:
uuid: "49d3c58ecd3844a3a859080b3a3d2837"
format_stamp: "Formatted at 2026-09-23 23:16:40 on dist-test-slave-754n"
server_key: "470b45b53ddeef2319454aadde1cc7f4"
server_key_iv: "5544a8137557af0c4510e53e8e554fdc"
server_key_version: "kuduclusterkey@0"
I20260923 23:16:40.591542 5312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ts-0/wal/instance:
uuid: "49d3c58ecd3844a3a859080b3a3d2837"
format_stamp: "Formatted at 2026-09-23 23:16:40 on dist-test-slave-754n"
server_key: "470b45b53ddeef2319454aadde1cc7f4"
server_key_iv: "5544a8137557af0c4510e53e8e554fdc"
server_key_version: "kuduclusterkey@0"
I20260923 23:16:40.597590 5312 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.001s
I20260923 23:16:40.620980 5329 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:16:40.621176 5312 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260923 23:16:40.621285 5312 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ts-0/wal
uuid: "49d3c58ecd3844a3a859080b3a3d2837"
format_stamp: "Formatted at 2026-09-23 23:16:40 on dist-test-slave-754n"
server_key: "470b45b53ddeef2319454aadde1cc7f4"
server_key_iv: "5544a8137557af0c4510e53e8e554fdc"
server_key_version: "kuduclusterkey@0"
I20260923 23:16:40.621402 5312 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:16:40.642858 5312 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:16:40.645170 5312 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:16:40.645340 5312 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:16:40.645555 5312 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 23:16:40.645927 5312 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 23:16:40.645980 5312 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 23:16:40.646047 5312 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 23:16:40.646092 5312 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 23:16:40.654109 5312 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.193:40957
I20260923 23:16:40.654487 5312 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ts-0/data/info.pb
I20260923 23:16:40.655017 5442 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.193:40957 every 8 connection(s)
I20260923 23:16:40.656752 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 5312
I20260923 23:16:40.656858 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ts-0/wal/instance
I20260923 23:16:40.657181 25487 external_mini_cluster.cc:1428] Setting key 6d216f9f17f4c509336f6087f436edde
Sep 23 23:16:40 dist-test-slave-754n krb5kdc[4755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205400, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.193@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
I20260923 23:16:40.668701 5445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:40.655184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:33391 (local address 127.24.227.193:60111)
0923 23:16:40.655523 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0923 23:16:40.655572 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0923 23:16:40.656330 (+ 758us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:16:40.657662 (+ 1332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:16:40.657667 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:16:40.657915 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:16:40.658179 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:16:40.658186 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:16:40.659203 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:16:40.659204 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:16:40.662091 (+ 2887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:16:40.662094 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:16:40.662654 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:16:40.662655 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:16:40.662800 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:16:40.663383 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:16:40.663394 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:16:40.665026 (+ 1632us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:16:40.667075 (+ 2049us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:16:40.667081 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:16:40.667083 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:16:40.667380 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:16:40.667607 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:16:40.667608 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:16:40.667608 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:16:40.667736 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:16:40.668050 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:16:40.668056 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:16:40.668215 (+ 159us) client_negotiation.cc:769] Sending connection context
0923 23:16:40.668312 (+ 97us) client_negotiation.cc:240] Negotiation successful
0923 23:16:40.668471 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":80,"threads_started":1}
I20260923 23:16:40.669255 5446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:40.655486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.193:60111 (local address 127.24.227.254:33391)
0923 23:16:40.655623 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0923 23:16:40.655625 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:16:40.655638 (+ 13us) server_negotiation.cc:407] Connection header received
0923 23:16:40.656513 (+ 875us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:16:40.656515 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:16:40.656530 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:16:40.657522 (+ 992us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:16:40.658294 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:40.659094 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:40.662189 (+ 3095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:40.662512 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:40.662573 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:16:40.665130 (+ 2557us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:16:40.665132 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:16:40.665132 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:16:40.665140 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:16:40.666984 (+ 1844us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:16:40.667475 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:16:40.667476 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:16:40.667477 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:16:40.667540 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:16:40.667802 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:16:40.667803 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:16:40.667804 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:16:40.667946 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:16:40.668023 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0923 23:16:40.669065 (+ 1042us) server_negotiation.cc:299] Negotiation successful
0923 23:16:40.669159 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":78,"threads_started":1}
I20260923 23:16:40.669993 5443 heartbeater.cc:347] Connected to a master server at 127.24.227.254:33391
I20260923 23:16:40.670083 5443 heartbeater.cc:464] Registering TS with master...
I20260923 23:16:40.670258 5443 heartbeater.cc:511] Master 127.24.227.254:33391 requested a full tablet report, sending...
I20260923 23:16:40.670838 5231 ts_manager.cc:194] Registered new tserver with Master: 49d3c58ecd3844a3a859080b3a3d2837 (127.24.227.193:40957)
I20260923 23:16:40.671737 5231 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.227.193@KRBTEST.COM'} at 127.24.227.193:60111
WARNING: no policy specified for kudu/127.24.227.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.227.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.227.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.194.keytab.
Entry for principal kudu/127.24.227.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.194.keytab.
I20260923 23:16:40.726177 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.227.212:35105/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.227.194:0
--local_ip_for_outbound_sockets=127.24.227.194
--webserver_interface=127.24.227.194
--webserver_port=0
--tserver_master_addrs=127.24.227.254:33391
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.194.keytab
--principal=kudu/127.24.227.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:41501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 23 23:16:40 dist-test-slave-754n krb5kdc[4755](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790205400, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260923 23:16:40.855926 5451 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:16:40.856268 5451 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:16:40.856353 5451 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:16:40.857977 5451 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:16:40.858078 5451 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:16:40.858158 5451 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.227.194
I20260923 23:16:40.860255 5451 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:41501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.24.227.212:35105/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.194.keytab
--principal=kudu/127.24.227.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.227.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.227.194
--webserver_port=0
--tserver_master_addrs=127.24.227.254:33391
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5451
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.227.194
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:16:40.860687 5451 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:16:40.861053 5451 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:16:40.867117 5456 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:16:40.867125 5457 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:16:40.867244 5459 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:16:40.867496 5451 server_base.cc:1034] running on GCE node
I20260923 23:16:40.867722 5451 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:16:40.867990 5451 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:16:40.869139 5451 hybrid_clock.cc:648] HybridClock initialized: now 1790205400869119 us; error 38 us; skew 500 ppm
Sep 23 23:16:40 dist-test-slave-754n krb5kdc[4755](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205400, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:16:40.872010 5451 init.cc:377] Logged in from keytab as kudu/127.24.227.194@KRBTEST.COM (short username kudu)
I20260923 23:16:40.872390 5451 webserver.cc:460] Webserver started at http://127.24.227.194:43899/ using document root <none> and password file <none>
I20260923 23:16:40.872627 5451 fs_manager.cc:359] Metadata directory not provided
I20260923 23:16:40.872694 5451 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:16:40.872845 5451 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 23 23:16:40 dist-test-slave-754n krb5kdc[4755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205400, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.194@KRBTEST.COM for HTTP/127.24.227.212@KRBTEST.COM
I20260923 23:16:40.885804 5451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ts-1/data/instance:
uuid: "745d4b316d984514bf8648dc6f19369e"
format_stamp: "Formatted at 2026-09-23 23:16:40 on dist-test-slave-754n"
server_key: "1aea6e68b546ff262e996485d2e581bf"
server_key_iv: "649735ec8ac360f70ee4febe42a66cb8"
server_key_version: "kuduclusterkey@0"
I20260923 23:16:40.886245 5451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ts-1/wal/instance:
uuid: "745d4b316d984514bf8648dc6f19369e"
format_stamp: "Formatted at 2026-09-23 23:16:40 on dist-test-slave-754n"
server_key: "1aea6e68b546ff262e996485d2e581bf"
server_key_iv: "649735ec8ac360f70ee4febe42a66cb8"
server_key_version: "kuduclusterkey@0"
I20260923 23:16:40.888051 5451 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260923 23:16:40.900774 5467 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:16:40.900988 5451 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 23:16:40.901058 5451 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ts-1/wal
uuid: "745d4b316d984514bf8648dc6f19369e"
format_stamp: "Formatted at 2026-09-23 23:16:40 on dist-test-slave-754n"
server_key: "1aea6e68b546ff262e996485d2e581bf"
server_key_iv: "649735ec8ac360f70ee4febe42a66cb8"
server_key_version: "kuduclusterkey@0"
I20260923 23:16:40.901125 5451 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:16:40.915097 5451 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:16:40.917384 5451 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:16:40.917541 5451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:16:40.917804 5451 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 23:16:40.918188 5451 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 23:16:40.918241 5451 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 23:16:40.918305 5451 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 23:16:40.918335 5451 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 23:16:40.925675 5451 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.194:44393
I20260923 23:16:40.926131 5451 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ts-1/data/info.pb
I20260923 23:16:40.926573 5580 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.194:44393 every 8 connection(s)
I20260923 23:16:40.932821 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 5451
I20260923 23:16:40.932938 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ts-1/wal/instance
I20260923 23:16:40.933148 25487 external_mini_cluster.cc:1428] Setting key 30c044429f6cd50c04b34eaff8cfab95
Sep 23 23:16:40 dist-test-slave-754n krb5kdc[4755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205400, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.194@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
I20260923 23:16:40.937590 5446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:40.927506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.194:56049 (local address 127.24.227.254:33391)
0923 23:16:40.927551 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0923 23:16:40.927554 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 23:16:40.927960 (+ 406us) server_negotiation.cc:407] Connection header received
0923 23:16:40.928690 (+ 730us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:16:40.928692 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:16:40.928710 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:16:40.928743 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:16:40.929548 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:40.930059 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:40.931158 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:40.931480 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:40.931528 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:16:40.934146 (+ 2618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:16:40.934147 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:16:40.934147 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:16:40.934154 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:16:40.935969 (+ 1815us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:16:40.936438 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:16:40.936439 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:16:40.936440 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:16:40.936499 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:16:40.936784 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:16:40.936785 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:16:40.936785 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:16:40.936925 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:16:40.936968 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0923 23:16:40.937369 (+ 401us) server_negotiation.cc:299] Negotiation successful
0923 23:16:40.937468 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 23:16:40.938375 5583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:40.927235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:33391 (local address 127.24.227.194:56049)
0923 23:16:40.927800 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0923 23:16:40.927851 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0923 23:16:40.928528 (+ 677us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:16:40.928945 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:16:40.928949 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:16:40.929176 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:16:40.929439 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:16:40.929444 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:16:40.930159 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:16:40.930160 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:16:40.931055 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:16:40.931058 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:16:40.931630 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:16:40.931631 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:16:40.931784 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:16:40.932377 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:16:40.932389 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:16:40.934042 (+ 1653us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:16:40.936062 (+ 2020us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:16:40.936069 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:16:40.936071 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:16:40.936344 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:16:40.936571 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:16:40.936572 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:16:40.936572 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:16:40.936704 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:16:40.937018 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:16:40.937025 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:16:40.937192 (+ 167us) client_negotiation.cc:769] Sending connection context
0923 23:16:40.937953 (+ 761us) client_negotiation.cc:240] Negotiation successful
0923 23:16:40.938119 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":80,"threads_started":1}
I20260923 23:16:40.938812 5581 heartbeater.cc:347] Connected to a master server at 127.24.227.254:33391
I20260923 23:16:40.938935 5581 heartbeater.cc:464] Registering TS with master...
I20260923 23:16:40.939110 5581 heartbeater.cc:511] Master 127.24.227.254:33391 requested a full tablet report, sending...
I20260923 23:16:40.939503 5231 ts_manager.cc:194] Registered new tserver with Master: 745d4b316d984514bf8648dc6f19369e (127.24.227.194:44393)
I20260923 23:16:40.939918 5231 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.227.194@KRBTEST.COM'} at 127.24.227.194:56049
WARNING: no policy specified for kudu/127.24.227.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.227.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.227.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.195.keytab.
Entry for principal kudu/127.24.227.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.195.keytab.
I20260923 23:16:40.966300 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.227.212:35105/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.227.195:0
--local_ip_for_outbound_sockets=127.24.227.195
--webserver_interface=127.24.227.195
--webserver_port=0
--tserver_master_addrs=127.24.227.254:33391
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.195.keytab
--principal=kudu/127.24.227.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:41501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:16:41.059480 5586 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:16:41.059844 5586 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:16:41.059966 5586 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:16:41.061517 5586 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:16:41.061622 5586 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:16:41.061712 5586 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.227.195
I20260923 23:16:41.063827 5586 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:41501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.24.227.212:35105/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.195.keytab
--principal=kudu/127.24.227.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.227.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.227.195
--webserver_port=0
--tserver_master_addrs=127.24.227.254:33391
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5586
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.227.195
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:16:41.064301 5586 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:16:41.064633 5586 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:16:41.071172 5592 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:16:41.071575 5594 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:16:41.071856 5586 server_base.cc:1034] running on GCE node
W20260923 23:16:41.072010 5591 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:16:41.072263 5586 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:16:41.072540 5586 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:16:41.073701 5586 hybrid_clock.cc:648] HybridClock initialized: now 1790205401073685 us; error 34 us; skew 500 ppm
Sep 23 23:16:41 dist-test-slave-754n krb5kdc[4755](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205401, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:16:41.077138 5586 init.cc:377] Logged in from keytab as kudu/127.24.227.195@KRBTEST.COM (short username kudu)
I20260923 23:16:41.080303 5586 webserver.cc:460] Webserver started at http://127.24.227.195:41917/ using document root <none> and password file <none>
I20260923 23:16:41.080585 5586 fs_manager.cc:359] Metadata directory not provided
I20260923 23:16:41.080643 5586 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:16:41.080770 5586 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 23 23:16:41 dist-test-slave-754n krb5kdc[4755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205401, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.195@KRBTEST.COM for HTTP/127.24.227.212@KRBTEST.COM
I20260923 23:16:41.093994 5586 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ts-2/data/instance:
uuid: "1657f5228f09409887cdca7e6d3a0ddb"
format_stamp: "Formatted at 2026-09-23 23:16:41 on dist-test-slave-754n"
server_key: "b5a99db4bf2b9a51403f2ca6fe419fc5"
server_key_iv: "d24263d4fa1e692fd1de3ceff671ad89"
server_key_version: "kuduclusterkey@0"
I20260923 23:16:41.094545 5586 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ts-2/wal/instance:
uuid: "1657f5228f09409887cdca7e6d3a0ddb"
format_stamp: "Formatted at 2026-09-23 23:16:41 on dist-test-slave-754n"
server_key: "b5a99db4bf2b9a51403f2ca6fe419fc5"
server_key_iv: "d24263d4fa1e692fd1de3ceff671ad89"
server_key_version: "kuduclusterkey@0"
I20260923 23:16:41.100957 5586 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.002s
I20260923 23:16:41.127686 5601 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:16:41.128060 5586 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260923 23:16:41.128181 5586 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ts-2/wal
uuid: "1657f5228f09409887cdca7e6d3a0ddb"
format_stamp: "Formatted at 2026-09-23 23:16:41 on dist-test-slave-754n"
server_key: "b5a99db4bf2b9a51403f2ca6fe419fc5"
server_key_iv: "d24263d4fa1e692fd1de3ceff671ad89"
server_key_version: "kuduclusterkey@0"
I20260923 23:16:41.128271 5586 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:16:41.142321 5586 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:16:41.144718 5586 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:16:41.144886 5586 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:16:41.145153 5586 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 23:16:41.145515 5586 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 23:16:41.145574 5586 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 23:16:41.145642 5586 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 23:16:41.145670 5586 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 23:16:41.153184 5586 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.195:41375
I20260923 23:16:41.153616 5586 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ts-2/data/info.pb
I20260923 23:16:41.154369 5714 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.195:41375 every 8 connection(s)
Sep 23 23:16:41 dist-test-slave-754n krb5kdc[4755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205401, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.195@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
I20260923 23:16:41.162073 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 5586
I20260923 23:16:41.162184 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ts-2/wal/instance
I20260923 23:16:41.162386 25487 external_mini_cluster.cc:1428] Setting key 9f83b79e9501b07b6a15068cd46bb5ef
I20260923 23:16:41.167688 5446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:41.156121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.195:46503 (local address 127.24.227.254:33391)
0923 23:16:41.156166 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0923 23:16:41.156167 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 23:16:41.156176 (+ 9us) server_negotiation.cc:407] Connection header received
0923 23:16:41.156203 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:16:41.156203 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:16:41.156213 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:16:41.156278 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:16:41.157046 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:41.157864 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:41.159373 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:41.159664 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:41.159709 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:16:41.163727 (+ 4018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:16:41.163729 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:16:41.163729 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:16:41.163737 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:16:41.165588 (+ 1851us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:16:41.166076 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:16:41.166077 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:16:41.166077 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:16:41.166141 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:16:41.166472 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:16:41.166473 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:16:41.166473 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:16:41.166610 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:16:41.166698 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0923 23:16:41.167500 (+ 802us) server_negotiation.cc:299] Negotiation successful
0923 23:16:41.167590 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260923 23:16:41.167845 5717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:41.154981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:33391 (local address 127.24.227.195:46503)
0923 23:16:41.155154 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0923 23:16:41.155201 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0923 23:16:41.155820 (+ 619us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:16:41.156437 (+ 617us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:16:41.156442 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:16:41.156667 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:16:41.156929 (+ 262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:16:41.156935 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:16:41.157982 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:16:41.157983 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:16:41.159270 (+ 1287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:16:41.159273 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:16:41.159811 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:16:41.159812 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:16:41.159972 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:16:41.160575 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:16:41.160585 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:16:41.162290 (+ 1705us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:16:41.165677 (+ 3387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:16:41.165684 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:16:41.165686 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:16:41.165992 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:16:41.166242 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:16:41.166243 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:16:41.166243 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:16:41.166396 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:16:41.166703 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:16:41.166709 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:16:41.167314 (+ 605us) client_negotiation.cc:769] Sending connection context
0923 23:16:41.167458 (+ 144us) client_negotiation.cc:240] Negotiation successful
0923 23:16:41.167615 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":80,"threads_started":1}
I20260923 23:16:41.168358 5715 heartbeater.cc:347] Connected to a master server at 127.24.227.254:33391
I20260923 23:16:41.168469 5715 heartbeater.cc:464] Registering TS with master...
I20260923 23:16:41.168649 5715 heartbeater.cc:511] Master 127.24.227.254:33391 requested a full tablet report, sending...
I20260923 23:16:41.169008 5231 ts_manager.cc:194] Registered new tserver with Master: 1657f5228f09409887cdca7e6d3a0ddb (127.24.227.195:41375)
I20260923 23:16:41.169507 5231 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.227.195@KRBTEST.COM'} at 127.24.227.195:46503
I20260923 23:16:41.174124 25487 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 23:16:41.174867 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 5312
I20260923 23:16:41.181701 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 5451
I20260923 23:16:41.188130 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 5586
I20260923 23:16:41.193603 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 5199
I20260923 23:16:41.198745 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.254:33391
--webserver_interface=127.24.227.254
--webserver_port=42859
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab
--principal=kudu/127.24.227.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:41501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.227.254:33391
--ranger_config_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.227.212:35105/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.24.227.212:35105/kms with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:16:41.278482 5718 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:16:41.278666 5718 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:16:41.278687 5718 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:16:41.279834 5718 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 23:16:41.279887 5718 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:16:41.279903 5718 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:16:41.279949 5718 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 23:16:41.279976 5718 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 23:16:41.281518 5718 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:41501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.24.227.212:35105/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.24.227.254:33391
--ranger_config_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab
--principal=kudu/127.24.227.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.254:33391
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.227.254
--webserver_port=42859
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5718
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:16:41.281741 5718 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:16:41.281963 5718 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:16:41.286641 5723 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:16:41.286654 5724 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:16:41.286741 5718 server_base.cc:1034] running on GCE node
W20260923 23:16:41.286643 5726 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:16:41.287096 5718 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:16:41.287313 5718 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:16:41.288450 5718 hybrid_clock.cc:648] HybridClock initialized: now 1790205401288418 us; error 50 us; skew 500 ppm
Sep 23 23:16:41 dist-test-slave-754n krb5kdc[4755](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205401, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:16:41.290784 5718 init.cc:377] Logged in from keytab as kudu/127.24.227.254@KRBTEST.COM (short username kudu)
I20260923 23:16:41.291203 5718 webserver.cc:460] Webserver started at http://127.24.227.254:42859/ using document root <none> and password file <none>
I20260923 23:16:41.291386 5718 fs_manager.cc:359] Metadata directory not provided
I20260923 23:16:41.291432 5718 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 23 23:16:41 dist-test-slave-754n krb5kdc[4755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205401, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.254@KRBTEST.COM for HTTP/127.24.227.212@KRBTEST.COM
I20260923 23:16:41.336524 5718 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 23:16:41.337309 5734 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:16:41.337543 5718 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 23:16:41.337639 5718 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/master-0/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/master-0/wal
uuid: "f2373ee6b47c4e8d9da1739e78341854"
format_stamp: "Formatted at 2026-09-23 23:16:39 on dist-test-slave-754n"
server_key: "0fc2e01f27d3593af043bafe76c7888f"
server_key_iv: "0ccc940fc251408dc74b4d9bb8386459"
server_key_version: "kuduclusterkey@0"
I20260923 23:16:41.338088 5718 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:16:41.351967 5718 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:16:41.353427 5718 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:16:41.353533 5718 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:16:41.357642 5718 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.254:33391
I20260923 23:16:41.357700 5786 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.254:33391 every 8 connection(s)
I20260923 23:16:41.358062 5718 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
I20260923 23:16:41.359165 5787 sys_catalog.cc:263] Verifying existing consensus state
I20260923 23:16:41.359925 5787 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2373ee6b47c4e8d9da1739e78341854: Bootstrap starting.
I20260923 23:16:41.364187 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 5718
I20260923 23:16:41.364494 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.227.212:35105/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.227.193:40957
--local_ip_for_outbound_sockets=127.24.227.193
--tserver_master_addrs=127.24.227.254:33391
--webserver_port=44555
--webserver_interface=127.24.227.193
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.193.keytab
--principal=kudu/127.24.227.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:41501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.24.227.212:35105/kms with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 23:16:41.367702 5787 log.cc:824] T 00000000000000000000000000000000 P f2373ee6b47c4e8d9da1739e78341854: Log is configured to *not* fsync() on all Append() calls
I20260923 23:16:41.369259 5787 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2373ee6b47c4e8d9da1739e78341854: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 23:16:41.369527 5787 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2373ee6b47c4e8d9da1739e78341854: Bootstrap complete.
I20260923 23:16:41.371948 5787 raft_consensus.cc:348] T 00000000000000000000000000000000 P f2373ee6b47c4e8d9da1739e78341854 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2373ee6b47c4e8d9da1739e78341854" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 33391 } }
I20260923 23:16:41.372212 5787 raft_consensus.cc:729] T 00000000000000000000000000000000 P f2373ee6b47c4e8d9da1739e78341854 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2373ee6b47c4e8d9da1739e78341854, State: Initialized, Role: FOLLOWER
I20260923 23:16:41.372390 5787 consensus_queue.cc:260] T 00000000000000000000000000000000 P f2373ee6b47c4e8d9da1739e78341854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2373ee6b47c4e8d9da1739e78341854" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 33391 } }
I20260923 23:16:41.372465 5787 raft_consensus.cc:388] T 00000000000000000000000000000000 P f2373ee6b47c4e8d9da1739e78341854 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 23:16:41.372517 5787 raft_consensus.cc:482] T 00000000000000000000000000000000 P f2373ee6b47c4e8d9da1739e78341854 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 23:16:41.372563 5787 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f2373ee6b47c4e8d9da1739e78341854 [term 1 FOLLOWER]: Advancing to term 2
I20260923 23:16:41.373504 5787 raft_consensus.cc:504] T 00000000000000000000000000000000 P f2373ee6b47c4e8d9da1739e78341854 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2373ee6b47c4e8d9da1739e78341854" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 33391 } }
I20260923 23:16:41.373617 5787 leader_election.cc:302] T 00000000000000000000000000000000 P f2373ee6b47c4e8d9da1739e78341854 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f2373ee6b47c4e8d9da1739e78341854; no voters:
I20260923 23:16:41.373796 5787 leader_election.cc:288] T 00000000000000000000000000000000 P f2373ee6b47c4e8d9da1739e78341854 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260923 23:16:41.373826 5791 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f2373ee6b47c4e8d9da1739e78341854 [term 2 FOLLOWER]: Leader election won for term 2
I20260923 23:16:41.373956 5791 raft_consensus.cc:686] T 00000000000000000000000000000000 P f2373ee6b47c4e8d9da1739e78341854 [term 2 LEADER]: Becoming Leader. State: Replica: f2373ee6b47c4e8d9da1739e78341854, State: Running, Role: LEADER
I20260923 23:16:41.374038 5791 consensus_queue.cc:237] T 00000000000000000000000000000000 P f2373ee6b47c4e8d9da1739e78341854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2373ee6b47c4e8d9da1739e78341854" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 33391 } }
I20260923 23:16:41.374064 5787 sys_catalog.cc:564] T 00000000000000000000000000000000 P f2373ee6b47c4e8d9da1739e78341854 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 23:16:41.374292 5793 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2373ee6b47c4e8d9da1739e78341854 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "f2373ee6b47c4e8d9da1739e78341854" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2373ee6b47c4e8d9da1739e78341854" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 33391 } } }
I20260923 23:16:41.374393 5793 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2373ee6b47c4e8d9da1739e78341854 [sys.catalog]: This master's current role is: LEADER
I20260923 23:16:41.374584 5792 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2373ee6b47c4e8d9da1739e78341854 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f2373ee6b47c4e8d9da1739e78341854. Latest consensus state: current_term: 2 leader_uuid: "f2373ee6b47c4e8d9da1739e78341854" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2373ee6b47c4e8d9da1739e78341854" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 33391 } } }
I20260923 23:16:41.374657 5792 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2373ee6b47c4e8d9da1739e78341854 [sys.catalog]: This master's current role is: LEADER
I20260923 23:16:41.374722 5795 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 23:16:41.375206 5795 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 23:16:41.375349 5795 catalog_manager.cc:1259] Loaded cluster ID: 17a85784336748afba6d6d4ba6627809
I20260923 23:16:41.375388 5795 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 23:16:41.375808 5787 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260923 23:16:41.376590 5795 catalog_manager.cc:1504] Loading token signing keys...
I20260923 23:16:41.376765 5795 catalog_manager.cc:5879] T 00000000000000000000000000000000 P f2373ee6b47c4e8d9da1739e78341854: Loaded TSK: 0
I20260923 23:16:41.376973 5795 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260923 23:16:41.475796 5789 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:16:41.476059 5789 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:16:41.476094 5789 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:16:41.477684 5789 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:16:41.477733 5789 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:16:41.477769 5789 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.227.193
I20260923 23:16:41.487285 5789 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:41501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.24.227.212:35105/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.193.keytab
--principal=kudu/127.24.227.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.227.193:40957
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.227.193
--webserver_port=44555
--tserver_master_addrs=127.24.227.254:33391
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5789
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.227.193
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:16:41.487574 5789 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:16:41.487852 5789 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:16:41.494925 5825 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:16:41.494956 5827 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:16:41.495512 5789 server_base.cc:1034] running on GCE node
W20260923 23:16:41.496016 5824 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:16:41.496273 5789 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:16:41.496568 5789 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:16:41.503923 5789 hybrid_clock.cc:648] HybridClock initialized: now 1790205401503889 us; error 46 us; skew 500 ppm
Sep 23 23:16:41 dist-test-slave-754n krb5kdc[4755](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205401, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:16:41.507290 5789 init.cc:377] Logged in from keytab as kudu/127.24.227.193@KRBTEST.COM (short username kudu)
I20260923 23:16:41.507824 5789 webserver.cc:460] Webserver started at http://127.24.227.193:44555/ using document root <none> and password file <none>
I20260923 23:16:41.508164 5789 fs_manager.cc:359] Metadata directory not provided
I20260923 23:16:41.508273 5789 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 23 23:16:41 dist-test-slave-754n krb5kdc[4755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205401, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.193@KRBTEST.COM for HTTP/127.24.227.212@KRBTEST.COM
I20260923 23:16:41.524718 5789 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 23:16:41.525771 5835 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:16:41.526001 5789 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260923 23:16:41.526104 5789 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ts-0/wal
uuid: "49d3c58ecd3844a3a859080b3a3d2837"
format_stamp: "Formatted at 2026-09-23 23:16:40 on dist-test-slave-754n"
server_key: "470b45b53ddeef2319454aadde1cc7f4"
server_key_iv: "5544a8137557af0c4510e53e8e554fdc"
server_key_version: "kuduclusterkey@0"
I20260923 23:16:41.526564 5789 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:16:41.545003 5789 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:16:41.547235 5789 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:16:41.547397 5789 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:16:41.547634 5789 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 23:16:41.548030 5789 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 23:16:41.548081 5789 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 23:16:41.548121 5789 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 23:16:41.548149 5789 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 23:16:41.554298 5789 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.193:40957
I20260923 23:16:41.554726 5789 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ts-0/data/info.pb
I20260923 23:16:41.554980 5948 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.193:40957 every 8 connection(s)
I20260923 23:16:41.559979 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 5789
I20260923 23:16:41.560240 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.227.212:35105/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.227.194:44393
--local_ip_for_outbound_sockets=127.24.227.194
--tserver_master_addrs=127.24.227.254:33391
--webserver_port=43899
--webserver_interface=127.24.227.194
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.194.keytab
--principal=kudu/127.24.227.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:41501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.24.227.212:35105/kms with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 23 23:16:41 dist-test-slave-754n krb5kdc[4755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205401, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.193@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
I20260923 23:16:41.578925 5951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:41.555679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:33391 (local address 127.24.227.193:54059)
0923 23:16:41.556219 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0923 23:16:41.556261 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0923 23:16:41.556886 (+ 625us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:16:41.557542 (+ 656us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:16:41.557546 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:16:41.557747 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:16:41.557995 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:16:41.558001 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:16:41.559351 (+ 1350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:16:41.559352 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:16:41.560507 (+ 1155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:16:41.560510 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:16:41.561460 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:16:41.561461 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:16:41.561618 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:16:41.562153 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:16:41.562164 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:16:41.574584 (+ 12420us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:16:41.577148 (+ 2564us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:16:41.577154 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:16:41.577156 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:16:41.577446 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:16:41.577713 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:16:41.577714 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:16:41.577714 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:16:41.577826 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:16:41.578342 (+ 516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:16:41.578347 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:16:41.578485 (+ 138us) client_negotiation.cc:769] Sending connection context
0923 23:16:41.578568 (+ 83us) client_negotiation.cc:240] Negotiation successful
0923 23:16:41.578709 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":80,"threads_started":1}
I20260923 23:16:41.580044 5952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:41.556037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.193:54059 (local address 127.24.227.254:33391)
0923 23:16:41.557085 (+ 1048us) server_negotiation.cc:206] Beginning negotiation
0923 23:16:41.557093 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 23:16:41.557126 (+ 33us) server_negotiation.cc:407] Connection header received
0923 23:16:41.557260 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:16:41.557266 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:16:41.557332 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:16:41.557464 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:16:41.558131 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:41.559234 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:41.560606 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:41.561249 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:41.561354 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:16:41.574727 (+ 13373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:16:41.574733 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:16:41.574733 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:16:41.574745 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:16:41.577028 (+ 2283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:16:41.577531 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:16:41.577535 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:16:41.577535 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:16:41.577640 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:16:41.577892 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:16:41.577893 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:16:41.577893 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:16:41.578250 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:16:41.579616 (+ 1366us) server_negotiation.cc:1035] Waiting for connection context
0923 23:16:41.579687 (+ 71us) server_negotiation.cc:299] Negotiation successful
0923 23:16:41.579802 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":998,"thread_start_us":72,"threads_started":1}
I20260923 23:16:41.580938 5949 heartbeater.cc:347] Connected to a master server at 127.24.227.254:33391
I20260923 23:16:41.581017 5949 heartbeater.cc:464] Registering TS with master...
I20260923 23:16:41.581179 5949 heartbeater.cc:511] Master 127.24.227.254:33391 requested a full tablet report, sending...
W20260923 23:16:41.582046 5949 heartbeater.cc:644] Failed to heartbeat to 127.24.227.254:33391 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260923 23:16:41.582110 5949 heartbeater.cc:464] Registering TS with master...
I20260923 23:16:41.582165 5949 heartbeater.cc:511] Master 127.24.227.254:33391 requested a full tablet report, sending...
I20260923 23:16:41.582563 5949 heartbeater.cc:464] Registering TS with master...
I20260923 23:16:41.582635 5949 heartbeater.cc:511] Master 127.24.227.254:33391 requested a full tablet report, sending...
W20260923 23:16:41.583014 5949 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260923 23:16:41.668555 5953 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:16:41.668753 5953 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:16:41.668776 5953 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:16:41.669972 5953 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:16:41.670012 5953 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:16:41.670038 5953 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.227.194
I20260923 23:16:41.671458 5953 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:41501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.24.227.212:35105/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.194.keytab
--principal=kudu/127.24.227.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.227.194:44393
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.227.194
--webserver_port=43899
--tserver_master_addrs=127.24.227.254:33391
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5953
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.227.194
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:16:41.671687 5953 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:16:41.671967 5953 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:16:41.683454 5961 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:16:41.683637 5959 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:16:41.684195 5958 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:16:41.684656 5953 server_base.cc:1034] running on GCE node
I20260923 23:16:41.684911 5953 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:16:41.685163 5953 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:16:41.701081 5953 hybrid_clock.cc:648] HybridClock initialized: now 1790205401701039 us; error 62 us; skew 500 ppm
Sep 23 23:16:41 dist-test-slave-754n krb5kdc[4755](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205401, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:16:41.704785 5953 init.cc:377] Logged in from keytab as kudu/127.24.227.194@KRBTEST.COM (short username kudu)
I20260923 23:16:41.705392 5953 webserver.cc:460] Webserver started at http://127.24.227.194:43899/ using document root <none> and password file <none>
I20260923 23:16:41.705705 5953 fs_manager.cc:359] Metadata directory not provided
I20260923 23:16:41.705829 5953 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 23 23:16:41 dist-test-slave-754n krb5kdc[4755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205401, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.194@KRBTEST.COM for HTTP/127.24.227.212@KRBTEST.COM
I20260923 23:16:41.724768 5953 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260923 23:16:41.726317 5969 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:16:41.726498 5953 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260923 23:16:41.726583 5953 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ts-1/wal
uuid: "745d4b316d984514bf8648dc6f19369e"
format_stamp: "Formatted at 2026-09-23 23:16:40 on dist-test-slave-754n"
server_key: "1aea6e68b546ff262e996485d2e581bf"
server_key_iv: "649735ec8ac360f70ee4febe42a66cb8"
server_key_version: "kuduclusterkey@0"
I20260923 23:16:41.727766 5953 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:16:41.747951 5953 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:16:41.750445 5953 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:16:41.750645 5953 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:16:41.750882 5953 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 23:16:41.751253 5953 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 23:16:41.751303 5953 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 23:16:41.751348 5953 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 23:16:41.751379 5953 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 23:16:41.758742 5953 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.194:44393
I20260923 23:16:41.759222 5953 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ts-1/data/info.pb
I20260923 23:16:41.761475 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 5953
I20260923 23:16:41.761766 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.227.212:35105/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.227.195:41375
--local_ip_for_outbound_sockets=127.24.227.195
--tserver_master_addrs=127.24.227.254:33391
--webserver_port=41917
--webserver_interface=127.24.227.195
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.195.keytab
--principal=kudu/127.24.227.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:41501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.24.227.212:35105/kms with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 23:16:41.761898 6082 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.194:44393 every 8 connection(s)
Sep 23 23:16:41 dist-test-slave-754n krb5kdc[4755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205401, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.194@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
I20260923 23:16:41.774037 5952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:41.763228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.194:44171 (local address 127.24.227.254:33391)
0923 23:16:41.763275 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0923 23:16:41.763277 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:16:41.763289 (+ 12us) server_negotiation.cc:407] Connection header received
0923 23:16:41.763620 (+ 331us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:16:41.763621 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:16:41.763635 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:16:41.763670 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:16:41.764583 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:41.765506 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:41.767214 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:41.767593 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:41.767636 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:16:41.770382 (+ 2746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:16:41.770384 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:16:41.770385 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:16:41.770392 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:16:41.772182 (+ 1790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:16:41.772720 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:16:41.772721 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:16:41.772721 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:16:41.772783 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:16:41.773106 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:16:41.773108 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:16:41.773108 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:16:41.773267 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:16:41.773314 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0923 23:16:41.773770 (+ 456us) server_negotiation.cc:299] Negotiation successful
0923 23:16:41.773875 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260923 23:16:41.774928 6085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:41.762529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:33391 (local address 127.24.227.194:44171)
0923 23:16:41.762693 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0923 23:16:41.762736 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0923 23:16:41.763485 (+ 749us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:16:41.763862 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:16:41.763905 (+ 43us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:16:41.764164 (+ 259us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:16:41.764464 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:16:41.764470 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:16:41.765632 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:16:41.765633 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:16:41.767103 (+ 1470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:16:41.767106 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:16:41.767748 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:16:41.767749 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:16:41.767936 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:16:41.768533 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:16:41.768543 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:16:41.770269 (+ 1726us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:16:41.772268 (+ 1999us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:16:41.772274 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:16:41.772276 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:16:41.772635 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:16:41.772871 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:16:41.772872 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:16:41.772872 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:16:41.773011 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:16:41.773379 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:16:41.773384 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:16:41.773578 (+ 194us) client_negotiation.cc:769] Sending connection context
0923 23:16:41.774462 (+ 884us) client_negotiation.cc:240] Negotiation successful
0923 23:16:41.774649 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":94,"threads_started":1}
I20260923 23:16:41.775683 6083 heartbeater.cc:347] Connected to a master server at 127.24.227.254:33391
I20260923 23:16:41.775779 6083 heartbeater.cc:464] Registering TS with master...
I20260923 23:16:41.775986 6083 heartbeater.cc:511] Master 127.24.227.254:33391 requested a full tablet report, sending...
W20260923 23:16:41.776664 6083 heartbeater.cc:644] Failed to heartbeat to 127.24.227.254:33391 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260923 23:16:41.776734 6083 heartbeater.cc:464] Registering TS with master...
I20260923 23:16:41.776801 6083 heartbeater.cc:511] Master 127.24.227.254:33391 requested a full tablet report, sending...
I20260923 23:16:41.777160 6083 heartbeater.cc:464] Registering TS with master...
I20260923 23:16:41.777237 6083 heartbeater.cc:511] Master 127.24.227.254:33391 requested a full tablet report, sending...
W20260923 23:16:41.777567 6083 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260923 23:16:41.874856 6086 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:16:41.875118 6086 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:16:41.875157 6086 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:16:41.876690 6086 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:16:41.876749 6086 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:16:41.876794 6086 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.227.195
I20260923 23:16:41.878451 6086 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:41501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.24.227.212:35105/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.195.keytab
--principal=kudu/127.24.227.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.227.195:41375
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.227.195
--webserver_port=41917
--tserver_master_addrs=127.24.227.254:33391
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6086
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.227.195
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:16:41.878712 6086 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:16:41.878907 6086 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:16:41.888123 6093 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:16:41.888253 6092 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:16:41.888391 6095 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:16:41.888859 6086 server_base.cc:1034] running on GCE node
I20260923 23:16:41.889055 6086 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:16:41.889292 6086 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:16:41.892542 6086 hybrid_clock.cc:648] HybridClock initialized: now 1790205401892527 us; error 33 us; skew 500 ppm
Sep 23 23:16:41 dist-test-slave-754n krb5kdc[4755](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205401, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:16:41.895196 6086 init.cc:377] Logged in from keytab as kudu/127.24.227.195@KRBTEST.COM (short username kudu)
I20260923 23:16:41.895681 6086 webserver.cc:460] Webserver started at http://127.24.227.195:41917/ using document root <none> and password file <none>
I20260923 23:16:41.895982 6086 fs_manager.cc:359] Metadata directory not provided
I20260923 23:16:41.896081 6086 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 23 23:16:41 dist-test-slave-754n krb5kdc[4755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205401, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.195@KRBTEST.COM for HTTP/127.24.227.212@KRBTEST.COM
I20260923 23:16:41.912515 6086 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 23:16:41.913514 6103 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:16:41.913686 6086 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260923 23:16:41.913798 6086 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ts-2/wal
uuid: "1657f5228f09409887cdca7e6d3a0ddb"
format_stamp: "Formatted at 2026-09-23 23:16:41 on dist-test-slave-754n"
server_key: "b5a99db4bf2b9a51403f2ca6fe419fc5"
server_key_iv: "d24263d4fa1e692fd1de3ceff671ad89"
server_key_version: "kuduclusterkey@0"
I20260923 23:16:41.914242 6086 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:16:41.945955 6086 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:16:41.948441 6086 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:16:41.948609 6086 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:16:41.948870 6086 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 23:16:41.949239 6086 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 23:16:41.949301 6086 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 23:16:41.949340 6086 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 23:16:41.949360 6086 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 23:16:41.956920 6086 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.195:41375
I20260923 23:16:41.957034 6216 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.195:41375 every 8 connection(s)
I20260923 23:16:41.957383 6086 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ts-2/data/info.pb
Sep 23 23:16:41 dist-test-slave-754n krb5kdc[4755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205401, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.195@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
I20260923 23:16:41.964862 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 6086
I20260923 23:16:41.968088 5952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:41.958209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.195:42555 (local address 127.24.227.254:33391)
0923 23:16:41.958261 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0923 23:16:41.958263 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:16:41.958496 (+ 233us) server_negotiation.cc:407] Connection header received
0923 23:16:41.958972 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:16:41.958973 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:16:41.958986 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:16:41.959066 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:16:41.959565 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:41.960315 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:41.961449 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:41.961738 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:41.961813 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:16:41.964094 (+ 2281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:16:41.964096 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:16:41.964097 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:16:41.964104 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:16:41.966646 (+ 2542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:16:41.967083 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:16:41.967085 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:16:41.967085 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:16:41.967149 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:16:41.967408 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:16:41.967409 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:16:41.967410 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:16:41.967531 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:16:41.967609 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0923 23:16:41.967913 (+ 304us) server_negotiation.cc:299] Negotiation successful
0923 23:16:41.967997 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260923 23:16:41.968284 6219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:41.958186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:33391 (local address 127.24.227.195:42555)
0923 23:16:41.958352 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0923 23:16:41.958393 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0923 23:16:41.958847 (+ 454us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:16:41.959119 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:16:41.959122 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:16:41.959287 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:16:41.959467 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:16:41.959471 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:16:41.960409 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:16:41.960411 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:16:41.961336 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:16:41.961339 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:16:41.961883 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:16:41.961884 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:16:41.962039 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:16:41.962628 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:16:41.962636 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:16:41.963998 (+ 1362us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:16:41.966749 (+ 2751us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:16:41.966755 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:16:41.966757 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:16:41.966987 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:16:41.967216 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:16:41.967217 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:16:41.967217 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:16:41.967336 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:16:41.967599 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:16:41.967605 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:16:41.967761 (+ 156us) client_negotiation.cc:769] Sending connection context
0923 23:16:41.967875 (+ 114us) client_negotiation.cc:240] Negotiation successful
0923 23:16:41.968051 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":95,"threads_started":1}
I20260923 23:16:41.968883 6217 heartbeater.cc:347] Connected to a master server at 127.24.227.254:33391
I20260923 23:16:41.968988 6217 heartbeater.cc:464] Registering TS with master...
I20260923 23:16:41.969179 6217 heartbeater.cc:511] Master 127.24.227.254:33391 requested a full tablet report, sending...
W20260923 23:16:41.969841 6217 heartbeater.cc:644] Failed to heartbeat to 127.24.227.254:33391 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260923 23:16:41.969923 6217 heartbeater.cc:464] Registering TS with master...
I20260923 23:16:41.969996 6217 heartbeater.cc:511] Master 127.24.227.254:33391 requested a full tablet report, sending...
I20260923 23:16:41.970295 6217 heartbeater.cc:464] Registering TS with master...
I20260923 23:16:41.970366 6217 heartbeater.cc:511] Master 127.24.227.254:33391 requested a full tablet report, sending...
W20260923 23:16:41.970667 6217 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260923 23:16:41.980747 6221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:41.966125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47210 (local address 127.24.227.254:33391)
0923 23:16:41.966246 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0923 23:16:41.966248 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:16:41.968890 (+ 2642us) server_negotiation.cc:407] Connection header received
0923 23:16:41.968914 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:16:41.968915 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:16:41.968928 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:16:41.969079 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:16:41.970075 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:41.970820 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:41.971960 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:41.972263 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:41.972351 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:16:41.973681 (+ 1330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:16:41.973683 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:16:41.973683 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:16:41.973690 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:16:41.975292 (+ 1602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:16:41.979851 (+ 4559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:16:41.979852 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:16:41.979852 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:16:41.979934 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:16:41.980144 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:16:41.980145 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:16:41.980146 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:16:41.980311 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:16:41.980502 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0923 23:16:41.980555 (+ 53us) server_negotiation.cc:299] Negotiation successful
0923 23:16:41.980636 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":69,"threads_started":1}
Sep 23 23:16:42 dist-test-slave-754n krb5kdc[4755](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790205402, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:16:42.583196 5949 heartbeater.cc:464] Registering TS with master...
I20260923 23:16:42.583359 5949 heartbeater.cc:511] Master 127.24.227.254:33391 requested a full tablet report, sending...
I20260923 23:16:42.583765 5751 ts_manager.cc:194] Registered new tserver with Master: 49d3c58ecd3844a3a859080b3a3d2837 (127.24.227.193:40957)
I20260923 23:16:42.584503 5751 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.227.193@KRBTEST.COM'} at 127.24.227.193:54059
I20260923 23:16:42.777724 6083 heartbeater.cc:464] Registering TS with master...
I20260923 23:16:42.777872 6083 heartbeater.cc:511] Master 127.24.227.254:33391 requested a full tablet report, sending...
I20260923 23:16:42.778143 5750 ts_manager.cc:194] Registered new tserver with Master: 745d4b316d984514bf8648dc6f19369e (127.24.227.194:44393)
I20260923 23:16:42.778576 5750 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.227.194@KRBTEST.COM'} at 127.24.227.194:44171
Sep 23 23:16:42 dist-test-slave-754n krb5kdc[4755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205402, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.254@KRBTEST.COM for HTTP/127.24.227.212@KRBTEST.COM
I20260923 23:16:42.970824 6217 heartbeater.cc:464] Registering TS with master...
I20260923 23:16:42.970964 6217 heartbeater.cc:511] Master 127.24.227.254:33391 requested a full tablet report, sending...
I20260923 23:16:42.971292 5750 ts_manager.cc:194] Registered new tserver with Master: 1657f5228f09409887cdca7e6d3a0ddb (127.24.227.195:41375)
I20260923 23:16:42.971817 5750 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.227.195@KRBTEST.COM'} at 127.24.227.195:42555
I20260923 23:16:42.981294 25487 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 23:16:42.994041 6239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:42.986607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47218 (local address 127.24.227.254:33391)
0923 23:16:42.986726 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0923 23:16:42.986728 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:16:42.986762 (+ 34us) server_negotiation.cc:407] Connection header received
0923 23:16:42.986783 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:16:42.986783 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:16:42.986794 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:16:42.986854 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:16:42.987495 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:42.988287 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:42.989264 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:42.989582 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:42.989663 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:16:42.991035 (+ 1372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:16:42.991036 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:16:42.991037 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:16:42.991043 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:16:42.992815 (+ 1772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:16:42.993194 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:16:42.993195 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:16:42.993195 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:16:42.993261 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:16:42.993491 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:16:42.993492 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:16:42.993493 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:16:42.993628 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:16:42.993749 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0923 23:16:42.993865 (+ 116us) server_negotiation.cc:299] Negotiation successful
0923 23:16:42.993947 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":61,"threads_started":1}
I20260923 23:16:42.996390 5750 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47218:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 23:16:42.998586 5750 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 23:16:43.006184 6248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:43.001450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.195:41375 (local address 127.0.0.1:35436)
0923 23:16:43.001835 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0923 23:16:43.001875 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0923 23:16:43.001938 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:16:43.002234 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:16:43.002237 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:16:43.002239 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:16:43.002309 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:16:43.002311 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:16:43.003589 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:16:43.003590 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:16:43.004921 (+ 1331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:16:43.004924 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:16:43.005909 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:16:43.005910 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:16:43.005990 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:16:43.006005 (+ 15us) client_negotiation.cc:769] Sending connection context
0923 23:16:43.006055 (+ 50us) client_negotiation.cc:240] Negotiation successful
0923 23:16:43.006103 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":72,"threads_started":1}
I20260923 23:16:43.006814 6247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:43.001675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35436 (local address 127.24.227.195:41375)
0923 23:16:43.002060 (+ 385us) server_negotiation.cc:206] Beginning negotiation
0923 23:16:43.002065 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 23:16:43.002075 (+ 10us) server_negotiation.cc:407] Connection header received
0923 23:16:43.002108 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:16:43.002111 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:16:43.002125 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:16:43.002392 (+ 267us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:16:43.002465 (+ 73us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:43.003487 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:43.005026 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:43.005803 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:43.006612 (+ 809us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:16:43.006679 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0923 23:16:43.006705 (+ 26us) server_negotiation.cc:299] Negotiation successful
0923 23:16:43.006741 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":59,"threads_started":1}
I20260923 23:16:43.007637 6151 tablet_service.cc:1467] Processing CreateTablet for tablet 1a5f291abcfd4c3daff56bbb4b268988 (DEFAULT_TABLE table=test-table [id=f8d3e62bde884e929da9f04a240f1ea1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 23:16:43.008028 6151 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a5f291abcfd4c3daff56bbb4b268988. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:16:43.010703 6244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:43.001684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.194:44393 (local address 127.0.0.1:34424)
0923 23:16:43.006571 (+ 4887us) negotiation.cc:107] Waiting for socket to connect
0923 23:16:43.006580 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 23:16:43.006624 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:16:43.006886 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:16:43.006887 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:16:43.006889 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:16:43.006923 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:16:43.006925 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:16:43.008162 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:16:43.008163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:16:43.009546 (+ 1383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:16:43.009548 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:16:43.010503 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:16:43.010504 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:16:43.010560 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:16:43.010564 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 23:16:43.010604 (+ 40us) client_negotiation.cc:240] Negotiation successful
0923 23:16:43.010640 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4871,"thread_start_us":63,"threads_started":1}
I20260923 23:16:43.010993 6249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:43.001787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34424 (local address 127.24.227.194:44393)
0923 23:16:43.001937 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0923 23:16:43.001942 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 23:16:43.006683 (+ 4741us) server_negotiation.cc:407] Connection header received
0923 23:16:43.006724 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:16:43.006727 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:16:43.006742 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:16:43.006809 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:16:43.006991 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:43.008069 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:43.009628 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:43.010355 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:43.010421 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:16:43.010470 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0923 23:16:43.010846 (+ 376us) server_negotiation.cc:299] Negotiation successful
0923 23:16:43.010901 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":69,"threads_started":1}
I20260923 23:16:43.011644 6017 tablet_service.cc:1467] Processing CreateTablet for tablet 1a5f291abcfd4c3daff56bbb4b268988 (DEFAULT_TABLE table=test-table [id=f8d3e62bde884e929da9f04a240f1ea1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 23:16:43.012046 6017 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a5f291abcfd4c3daff56bbb4b268988. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:16:43.015189 6250 tablet_bootstrap.cc:492] T 1a5f291abcfd4c3daff56bbb4b268988 P 1657f5228f09409887cdca7e6d3a0ddb: Bootstrap starting.
I20260923 23:16:43.015830 6250 tablet_bootstrap.cc:654] T 1a5f291abcfd4c3daff56bbb4b268988 P 1657f5228f09409887cdca7e6d3a0ddb: Neither blocks nor log segments found. Creating new log.
I20260923 23:16:43.016093 6248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:43.001498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.193:40957 (local address 127.0.0.1:41372)
0923 23:16:43.006473 (+ 4975us) negotiation.cc:107] Waiting for socket to connect
0923 23:16:43.006484 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 23:16:43.006527 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:16:43.012351 (+ 5824us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:16:43.012352 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:16:43.012353 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:16:43.012386 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:16:43.012387 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:16:43.013756 (+ 1369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:16:43.013757 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:16:43.014803 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:16:43.014806 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:16:43.015838 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:16:43.015839 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:16:43.015923 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:16:43.015927 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 23:16:43.015982 (+ 55us) client_negotiation.cc:240] Negotiation successful
0923 23:16:43.016028 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4965,"thread_start_us":78,"threads_started":1}
I20260923 23:16:43.016177 6250 log.cc:824] T 1a5f291abcfd4c3daff56bbb4b268988 P 1657f5228f09409887cdca7e6d3a0ddb: Log is configured to *not* fsync() on all Append() calls
I20260923 23:16:43.016414 6246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:43.001594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41372 (local address 127.24.227.193:40957)
0923 23:16:43.012011 (+ 10417us) server_negotiation.cc:206] Beginning negotiation
0923 23:16:43.012016 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 23:16:43.012027 (+ 11us) server_negotiation.cc:407] Connection header received
0923 23:16:43.012061 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:16:43.012063 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:16:43.012078 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:16:43.012258 (+ 180us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:16:43.012584 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:43.013664 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:43.014908 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:43.015662 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:43.015744 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:16:43.015800 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0923 23:16:43.016261 (+ 461us) server_negotiation.cc:299] Negotiation successful
0923 23:16:43.016319 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10378,"thread_start_us":42,"threads_started":1}
I20260923 23:16:43.016906 6251 tablet_bootstrap.cc:492] T 1a5f291abcfd4c3daff56bbb4b268988 P 745d4b316d984514bf8648dc6f19369e: Bootstrap starting.
I20260923 23:16:43.016932 6250 tablet_bootstrap.cc:492] T 1a5f291abcfd4c3daff56bbb4b268988 P 1657f5228f09409887cdca7e6d3a0ddb: No bootstrap required, opened a new log
I20260923 23:16:43.016989 6250 ts_tablet_manager.cc:1397] T 1a5f291abcfd4c3daff56bbb4b268988 P 1657f5228f09409887cdca7e6d3a0ddb: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260923 23:16:43.017573 6251 tablet_bootstrap.cc:654] T 1a5f291abcfd4c3daff56bbb4b268988 P 745d4b316d984514bf8648dc6f19369e: Neither blocks nor log segments found. Creating new log.
I20260923 23:16:43.017897 6251 log.cc:824] T 1a5f291abcfd4c3daff56bbb4b268988 P 745d4b316d984514bf8648dc6f19369e: Log is configured to *not* fsync() on all Append() calls
I20260923 23:16:43.019161 6250 raft_consensus.cc:348] T 1a5f291abcfd4c3daff56bbb4b268988 P 1657f5228f09409887cdca7e6d3a0ddb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1657f5228f09409887cdca7e6d3a0ddb" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 41375 } } peers { permanent_uuid: "49d3c58ecd3844a3a859080b3a3d2837" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 40957 } } peers { permanent_uuid: "745d4b316d984514bf8648dc6f19369e" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 44393 } }
I20260923 23:16:43.019315 6250 raft_consensus.cc:374] T 1a5f291abcfd4c3daff56bbb4b268988 P 1657f5228f09409887cdca7e6d3a0ddb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:16:43.019357 6250 raft_consensus.cc:729] T 1a5f291abcfd4c3daff56bbb4b268988 P 1657f5228f09409887cdca7e6d3a0ddb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1657f5228f09409887cdca7e6d3a0ddb, State: Initialized, Role: FOLLOWER
I20260923 23:16:43.019465 6250 consensus_queue.cc:260] T 1a5f291abcfd4c3daff56bbb4b268988 P 1657f5228f09409887cdca7e6d3a0ddb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1657f5228f09409887cdca7e6d3a0ddb" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 41375 } } peers { permanent_uuid: "49d3c58ecd3844a3a859080b3a3d2837" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 40957 } } peers { permanent_uuid: "745d4b316d984514bf8648dc6f19369e" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 44393 } }
I20260923 23:16:43.019706 6250 ts_tablet_manager.cc:1428] T 1a5f291abcfd4c3daff56bbb4b268988 P 1657f5228f09409887cdca7e6d3a0ddb: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260923 23:16:43.019845 6217 heartbeater.cc:503] Master 127.24.227.254:33391 was elected leader, sending a full tablet report...
I20260923 23:16:43.020411 5873 tablet_service.cc:1467] Processing CreateTablet for tablet 1a5f291abcfd4c3daff56bbb4b268988 (DEFAULT_TABLE table=test-table [id=f8d3e62bde884e929da9f04a240f1ea1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 23:16:43.020716 5873 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a5f291abcfd4c3daff56bbb4b268988. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:16:43.023278 6255 tablet_bootstrap.cc:492] T 1a5f291abcfd4c3daff56bbb4b268988 P 49d3c58ecd3844a3a859080b3a3d2837: Bootstrap starting.
I20260923 23:16:43.023955 6255 tablet_bootstrap.cc:654] T 1a5f291abcfd4c3daff56bbb4b268988 P 49d3c58ecd3844a3a859080b3a3d2837: Neither blocks nor log segments found. Creating new log.
I20260923 23:16:43.024281 6255 log.cc:824] T 1a5f291abcfd4c3daff56bbb4b268988 P 49d3c58ecd3844a3a859080b3a3d2837: Log is configured to *not* fsync() on all Append() calls
I20260923 23:16:43.024438 6251 tablet_bootstrap.cc:492] T 1a5f291abcfd4c3daff56bbb4b268988 P 745d4b316d984514bf8648dc6f19369e: No bootstrap required, opened a new log
I20260923 23:16:43.024506 6251 ts_tablet_manager.cc:1397] T 1a5f291abcfd4c3daff56bbb4b268988 P 745d4b316d984514bf8648dc6f19369e: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20260923 23:16:43.025600 6255 tablet_bootstrap.cc:492] T 1a5f291abcfd4c3daff56bbb4b268988 P 49d3c58ecd3844a3a859080b3a3d2837: No bootstrap required, opened a new log
I20260923 23:16:43.025664 6255 ts_tablet_manager.cc:1397] T 1a5f291abcfd4c3daff56bbb4b268988 P 49d3c58ecd3844a3a859080b3a3d2837: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260923 23:16:43.027460 6251 raft_consensus.cc:348] T 1a5f291abcfd4c3daff56bbb4b268988 P 745d4b316d984514bf8648dc6f19369e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1657f5228f09409887cdca7e6d3a0ddb" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 41375 } } peers { permanent_uuid: "49d3c58ecd3844a3a859080b3a3d2837" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 40957 } } peers { permanent_uuid: "745d4b316d984514bf8648dc6f19369e" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 44393 } }
I20260923 23:16:43.027607 6251 raft_consensus.cc:374] T 1a5f291abcfd4c3daff56bbb4b268988 P 745d4b316d984514bf8648dc6f19369e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:16:43.027639 6251 raft_consensus.cc:729] T 1a5f291abcfd4c3daff56bbb4b268988 P 745d4b316d984514bf8648dc6f19369e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 745d4b316d984514bf8648dc6f19369e, State: Initialized, Role: FOLLOWER
I20260923 23:16:43.027770 6251 consensus_queue.cc:260] T 1a5f291abcfd4c3daff56bbb4b268988 P 745d4b316d984514bf8648dc6f19369e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1657f5228f09409887cdca7e6d3a0ddb" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 41375 } } peers { permanent_uuid: "49d3c58ecd3844a3a859080b3a3d2837" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 40957 } } peers { permanent_uuid: "745d4b316d984514bf8648dc6f19369e" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 44393 } }
I20260923 23:16:43.028046 6251 ts_tablet_manager.cc:1428] T 1a5f291abcfd4c3daff56bbb4b268988 P 745d4b316d984514bf8648dc6f19369e: Time spent starting tablet: real 0.003s user 0.000s sys 0.004s
I20260923 23:16:43.028036 6255 raft_consensus.cc:348] T 1a5f291abcfd4c3daff56bbb4b268988 P 49d3c58ecd3844a3a859080b3a3d2837 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1657f5228f09409887cdca7e6d3a0ddb" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 41375 } } peers { permanent_uuid: "49d3c58ecd3844a3a859080b3a3d2837" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 40957 } } peers { permanent_uuid: "745d4b316d984514bf8648dc6f19369e" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 44393 } }
I20260923 23:16:43.028163 6083 heartbeater.cc:503] Master 127.24.227.254:33391 was elected leader, sending a full tablet report...
I20260923 23:16:43.028179 6255 raft_consensus.cc:374] T 1a5f291abcfd4c3daff56bbb4b268988 P 49d3c58ecd3844a3a859080b3a3d2837 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:16:43.028218 6255 raft_consensus.cc:729] T 1a5f291abcfd4c3daff56bbb4b268988 P 49d3c58ecd3844a3a859080b3a3d2837 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49d3c58ecd3844a3a859080b3a3d2837, State: Initialized, Role: FOLLOWER
I20260923 23:16:43.028373 6255 consensus_queue.cc:260] T 1a5f291abcfd4c3daff56bbb4b268988 P 49d3c58ecd3844a3a859080b3a3d2837 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1657f5228f09409887cdca7e6d3a0ddb" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 41375 } } peers { permanent_uuid: "49d3c58ecd3844a3a859080b3a3d2837" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 40957 } } peers { permanent_uuid: "745d4b316d984514bf8648dc6f19369e" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 44393 } }
I20260923 23:16:43.028793 6255 ts_tablet_manager.cc:1428] T 1a5f291abcfd4c3daff56bbb4b268988 P 49d3c58ecd3844a3a859080b3a3d2837: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 23:16:43.028906 5949 heartbeater.cc:503] Master 127.24.227.254:33391 was elected leader, sending a full tablet report...
W20260923 23:16:43.056421 5950 tablet.cc:2369] T 1a5f291abcfd4c3daff56bbb4b268988 P 49d3c58ecd3844a3a859080b3a3d2837: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 23:16:43.090729 6254 raft_consensus.cc:482] T 1a5f291abcfd4c3daff56bbb4b268988 P 1657f5228f09409887cdca7e6d3a0ddb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 23:16:43.090883 6254 raft_consensus.cc:504] T 1a5f291abcfd4c3daff56bbb4b268988 P 1657f5228f09409887cdca7e6d3a0ddb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1657f5228f09409887cdca7e6d3a0ddb" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 41375 } } peers { permanent_uuid: "49d3c58ecd3844a3a859080b3a3d2837" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 40957 } } peers { permanent_uuid: "745d4b316d984514bf8648dc6f19369e" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 44393 } }
I20260923 23:16:43.091284 6254 leader_election.cc:288] T 1a5f291abcfd4c3daff56bbb4b268988 P 1657f5228f09409887cdca7e6d3a0ddb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49d3c58ecd3844a3a859080b3a3d2837 (127.24.227.193:40957), 745d4b316d984514bf8648dc6f19369e (127.24.227.194:44393)
I20260923 23:16:43.096937 6246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:43.091667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.195:36461 (local address 127.24.227.193:40957)
0923 23:16:43.091708 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0923 23:16:43.091710 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:16:43.093358 (+ 1648us) server_negotiation.cc:407] Connection header received
0923 23:16:43.093463 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:16:43.093465 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:16:43.093477 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:16:43.093528 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:16:43.093750 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:43.094509 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:43.095992 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:43.096537 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:43.096580 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:16:43.096588 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0923 23:16:43.096820 (+ 232us) server_negotiation.cc:299] Negotiation successful
0923 23:16:43.096857 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 23:16:43.097031 6249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:43.091616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.195:32799 (local address 127.24.227.194:44393)
0923 23:16:43.091669 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0923 23:16:43.091671 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:16:43.093535 (+ 1864us) server_negotiation.cc:407] Connection header received
0923 23:16:43.093631 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:16:43.093632 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:16:43.093642 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:16:43.093670 (+ 28us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:16:43.093868 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:43.094541 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:43.096150 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:43.096692 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:43.096727 (+ 35us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:16:43.096734 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0923 23:16:43.096927 (+ 193us) server_negotiation.cc:299] Negotiation successful
0923 23:16:43.096965 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260923 23:16:43.097241 6260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:43.091410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.193:40957 (local address 127.24.227.195:36461)
0923 23:16:43.093267 (+ 1857us) negotiation.cc:107] Waiting for socket to connect
0923 23:16:43.093278 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 23:16:43.093372 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:16:43.093581 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:16:43.093582 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:16:43.093584 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:16:43.093670 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:16:43.093672 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:16:43.094609 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:16:43.094610 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:16:43.095866 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:16:43.095869 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:16:43.096678 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:16:43.096679 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:16:43.096743 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:16:43.096747 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 23:16:43.097067 (+ 320us) client_negotiation.cc:240] Negotiation successful
0923 23:16:43.097109 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1840,"thread_start_us":86,"threads_started":1}
I20260923 23:16:43.097574 5884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a5f291abcfd4c3daff56bbb4b268988" candidate_uuid: "1657f5228f09409887cdca7e6d3a0ddb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49d3c58ecd3844a3a859080b3a3d2837" is_pre_election: true
I20260923 23:16:43.097725 5884 raft_consensus.cc:2454] T 1a5f291abcfd4c3daff56bbb4b268988 P 49d3c58ecd3844a3a859080b3a3d2837 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1657f5228f09409887cdca7e6d3a0ddb in term 0.
I20260923 23:16:43.097978 6105 leader_election.cc:302] T 1a5f291abcfd4c3daff56bbb4b268988 P 1657f5228f09409887cdca7e6d3a0ddb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1657f5228f09409887cdca7e6d3a0ddb, 49d3c58ecd3844a3a859080b3a3d2837; no voters:
I20260923 23:16:43.098119 6254 raft_consensus.cc:2781] T 1a5f291abcfd4c3daff56bbb4b268988 P 1657f5228f09409887cdca7e6d3a0ddb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 23:16:43.098176 6254 raft_consensus.cc:482] T 1a5f291abcfd4c3daff56bbb4b268988 P 1657f5228f09409887cdca7e6d3a0ddb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 23:16:43.098205 6254 raft_consensus.cc:3037] T 1a5f291abcfd4c3daff56bbb4b268988 P 1657f5228f09409887cdca7e6d3a0ddb [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:16:43.098853 6254 raft_consensus.cc:504] T 1a5f291abcfd4c3daff56bbb4b268988 P 1657f5228f09409887cdca7e6d3a0ddb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1657f5228f09409887cdca7e6d3a0ddb" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 41375 } } peers { permanent_uuid: "49d3c58ecd3844a3a859080b3a3d2837" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 40957 } } peers { permanent_uuid: "745d4b316d984514bf8648dc6f19369e" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 44393 } }
I20260923 23:16:43.098994 6254 leader_election.cc:288] T 1a5f291abcfd4c3daff56bbb4b268988 P 1657f5228f09409887cdca7e6d3a0ddb [CANDIDATE]: Term 1 election: Requested vote from peers 49d3c58ecd3844a3a859080b3a3d2837 (127.24.227.193:40957), 745d4b316d984514bf8648dc6f19369e (127.24.227.194:44393)
I20260923 23:16:43.098086 6261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:43.091534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.194:44393 (local address 127.24.227.195:32799)
0923 23:16:43.093481 (+ 1947us) negotiation.cc:107] Waiting for socket to connect
0923 23:16:43.093490 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 23:16:43.093558 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:16:43.093733 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:16:43.093734 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:16:43.093735 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:16:43.093784 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:16:43.093786 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:16:43.094901 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:16:43.094901 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:16:43.096067 (+ 1166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:16:43.096069 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:16:43.096782 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:16:43.096782 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:16:43.096859 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:16:43.096861 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 23:16:43.097954 (+ 1093us) client_negotiation.cc:240] Negotiation successful
0923 23:16:43.098017 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1932,"thread_start_us":83,"threads_started":1}
I20260923 23:16:43.099171 5884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a5f291abcfd4c3daff56bbb4b268988" candidate_uuid: "1657f5228f09409887cdca7e6d3a0ddb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49d3c58ecd3844a3a859080b3a3d2837"
I20260923 23:16:43.099251 5884 raft_consensus.cc:3037] T 1a5f291abcfd4c3daff56bbb4b268988 P 49d3c58ecd3844a3a859080b3a3d2837 [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:16:43.099465 6034 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a5f291abcfd4c3daff56bbb4b268988" candidate_uuid: "1657f5228f09409887cdca7e6d3a0ddb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "745d4b316d984514bf8648dc6f19369e" is_pre_election: true
I20260923 23:16:43.099471 6033 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a5f291abcfd4c3daff56bbb4b268988" candidate_uuid: "1657f5228f09409887cdca7e6d3a0ddb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "745d4b316d984514bf8648dc6f19369e"
I20260923 23:16:43.099624 6034 raft_consensus.cc:2454] T 1a5f291abcfd4c3daff56bbb4b268988 P 745d4b316d984514bf8648dc6f19369e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1657f5228f09409887cdca7e6d3a0ddb in term 0.
I20260923 23:16:43.100109 5884 raft_consensus.cc:2454] T 1a5f291abcfd4c3daff56bbb4b268988 P 49d3c58ecd3844a3a859080b3a3d2837 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1657f5228f09409887cdca7e6d3a0ddb in term 1.
I20260923 23:16:43.100306 6105 leader_election.cc:302] T 1a5f291abcfd4c3daff56bbb4b268988 P 1657f5228f09409887cdca7e6d3a0ddb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1657f5228f09409887cdca7e6d3a0ddb, 49d3c58ecd3844a3a859080b3a3d2837; no voters: 745d4b316d984514bf8648dc6f19369e
I20260923 23:16:43.100418 6254 raft_consensus.cc:2781] T 1a5f291abcfd4c3daff56bbb4b268988 P 1657f5228f09409887cdca7e6d3a0ddb [term 1 FOLLOWER]: Leader election won for term 1
I20260923 23:16:43.100541 6254 raft_consensus.cc:686] T 1a5f291abcfd4c3daff56bbb4b268988 P 1657f5228f09409887cdca7e6d3a0ddb [term 1 LEADER]: Becoming Leader. State: Replica: 1657f5228f09409887cdca7e6d3a0ddb, State: Running, Role: LEADER
I20260923 23:16:43.100649 6254 consensus_queue.cc:237] T 1a5f291abcfd4c3daff56bbb4b268988 P 1657f5228f09409887cdca7e6d3a0ddb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1657f5228f09409887cdca7e6d3a0ddb" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 41375 } } peers { permanent_uuid: "49d3c58ecd3844a3a859080b3a3d2837" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 40957 } } peers { permanent_uuid: "745d4b316d984514bf8648dc6f19369e" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 44393 } }
I20260923 23:16:43.101382 5750 catalog_manager.cc:5495] T 1a5f291abcfd4c3daff56bbb4b268988 P 1657f5228f09409887cdca7e6d3a0ddb reported cstate change: term changed from 0 to 1, leader changed from <none> to 1657f5228f09409887cdca7e6d3a0ddb (127.24.227.195). New cstate: current_term: 1 leader_uuid: "1657f5228f09409887cdca7e6d3a0ddb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1657f5228f09409887cdca7e6d3a0ddb" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 41375 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49d3c58ecd3844a3a859080b3a3d2837" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 40957 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "745d4b316d984514bf8648dc6f19369e" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 44393 } health_report { overall_health: UNKNOWN } } }
Sep 23 23:16:43 dist-test-slave-754n krb5kdc[4755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205402, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.254@KRBTEST.COM for HTTP/127.24.227.212@KRBTEST.COM
I20260923 23:16:43.120478 6247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:43.117509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35448 (local address 127.24.227.195:41375)
0923 23:16:43.117556 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0923 23:16:43.117558 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:16:43.117565 (+ 7us) server_negotiation.cc:407] Connection header received
0923 23:16:43.117597 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:16:43.117598 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:16:43.117615 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:16:43.117682 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 23:16:43.117854 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:43.118556 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:43.119348 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:43.119626 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:43.119707 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:16:43.119924 (+ 217us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 23:16:43.120035 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 23:16:43.120117 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0923 23:16:43.120395 (+ 278us) server_negotiation.cc:299] Negotiation successful
0923 23:16:43.120416 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260923 23:16:43.125450 6033 raft_consensus.cc:3037] T 1a5f291abcfd4c3daff56bbb4b268988 P 745d4b316d984514bf8648dc6f19369e [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:16:43.126472 6033 raft_consensus.cc:1261] T 1a5f291abcfd4c3daff56bbb4b268988 P 745d4b316d984514bf8648dc6f19369e [term 1 FOLLOWER]: Refusing update from remote peer 1657f5228f09409887cdca7e6d3a0ddb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 23:16:43.126762 5884 raft_consensus.cc:1261] T 1a5f291abcfd4c3daff56bbb4b268988 P 49d3c58ecd3844a3a859080b3a3d2837 [term 1 FOLLOWER]: Refusing update from remote peer 1657f5228f09409887cdca7e6d3a0ddb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 23:16:43.127076 6254 consensus_queue.cc:1035] T 1a5f291abcfd4c3daff56bbb4b268988 P 1657f5228f09409887cdca7e6d3a0ddb [LEADER]: Connected to new peer: Peer: permanent_uuid: "745d4b316d984514bf8648dc6f19369e" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 44393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 23:16:43.127275 6262 consensus_queue.cc:1035] T 1a5f291abcfd4c3daff56bbb4b268988 P 1657f5228f09409887cdca7e6d3a0ddb [LEADER]: Connected to new peer: Peer: permanent_uuid: "49d3c58ecd3844a3a859080b3a3d2837" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 40957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 23:16:43.131129 6265 mvcc.cc:205] Tried to move back new op lower bound from 7332681331199336448 to 7332681331100823552. Current Snapshot: MvccSnapshot[applied={T|T < 7332681331199336448}]
I20260923 23:16:43.132385 6266 mvcc.cc:205] Tried to move back new op lower bound from 7332681331199336448 to 7332681331100823552. Current Snapshot: MvccSnapshot[applied={T|T < 7332681331199336448}]
I20260923 23:16:43.136657 5751 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47218:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 23:16:43.136746 5751 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47218:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 23:16:43.137501 5751 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f2373ee6b47c4e8d9da1739e78341854: Sending DeleteTablet for 3 replicas of tablet 1a5f291abcfd4c3daff56bbb4b268988
I20260923 23:16:43.137840 6017 tablet_service.cc:1514] Processing DeleteTablet for tablet 1a5f291abcfd4c3daff56bbb4b268988 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 23:16:43 UTC) from {username='kudu', principal='kudu/127.24.227.254@KRBTEST.COM'} at 127.0.0.1:34424
I20260923 23:16:43.138151 5873 tablet_service.cc:1514] Processing DeleteTablet for tablet 1a5f291abcfd4c3daff56bbb4b268988 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 23:16:43 UTC) from {username='kudu', principal='kudu/127.24.227.254@KRBTEST.COM'} at 127.0.0.1:41372
I20260923 23:16:43.138382 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 5789
I20260923 23:16:43.142272 6151 tablet_service.cc:1514] Processing DeleteTablet for tablet 1a5f291abcfd4c3daff56bbb4b268988 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 23:16:43 UTC) from {username='kudu', principal='kudu/127.24.227.254@KRBTEST.COM'} at 127.0.0.1:35436
I20260923 23:16:43.148211 6275 tablet_replica.cc:330] T 1a5f291abcfd4c3daff56bbb4b268988 P 745d4b316d984514bf8648dc6f19369e: stopping tablet replica
I20260923 23:16:43.148325 6275 raft_consensus.cc:2229] T 1a5f291abcfd4c3daff56bbb4b268988 P 745d4b316d984514bf8648dc6f19369e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 23:16:43.148399 6275 raft_consensus.cc:2258] T 1a5f291abcfd4c3daff56bbb4b268988 P 745d4b316d984514bf8648dc6f19369e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 23:16:43.148545 6278 tablet_replica.cc:330] T 1a5f291abcfd4c3daff56bbb4b268988 P 1657f5228f09409887cdca7e6d3a0ddb: stopping tablet replica
I20260923 23:16:43.148629 6278 raft_consensus.cc:2229] T 1a5f291abcfd4c3daff56bbb4b268988 P 1657f5228f09409887cdca7e6d3a0ddb [term 1 LEADER]: Raft consensus shutting down.
I20260923 23:16:43.148710 6278 raft_consensus.cc:2258] T 1a5f291abcfd4c3daff56bbb4b268988 P 1657f5228f09409887cdca7e6d3a0ddb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 23:16:43.149288 6278 ts_tablet_manager.cc:1905] T 1a5f291abcfd4c3daff56bbb4b268988 P 1657f5228f09409887cdca7e6d3a0ddb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 23:16:43.151750 6275 ts_tablet_manager.cc:1905] T 1a5f291abcfd4c3daff56bbb4b268988 P 745d4b316d984514bf8648dc6f19369e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 23:16:43.153326 6278 ts_tablet_manager.cc:1918] T 1a5f291abcfd4c3daff56bbb4b268988 P 1657f5228f09409887cdca7e6d3a0ddb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 23:16:43.153398 6278 log.cc:1196] T 1a5f291abcfd4c3daff56bbb4b268988 P 1657f5228f09409887cdca7e6d3a0ddb: Deleting WAL directory at /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ts-2/wal/wals/1a5f291abcfd4c3daff56bbb4b268988
I20260923 23:16:43.153663 6278 ts_tablet_manager.cc:1939] T 1a5f291abcfd4c3daff56bbb4b268988 P 1657f5228f09409887cdca7e6d3a0ddb: Deleting consensus metadata
I20260923 23:16:43.154053 5735 catalog_manager.cc:4841] TS 1657f5228f09409887cdca7e6d3a0ddb (127.24.227.195:41375): tablet 1a5f291abcfd4c3daff56bbb4b268988 (table test-table [id=f8d3e62bde884e929da9f04a240f1ea1]) successfully deleted
I20260923 23:16:43.154214 6275 ts_tablet_manager.cc:1918] T 1a5f291abcfd4c3daff56bbb4b268988 P 745d4b316d984514bf8648dc6f19369e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 23:16:43.154280 6275 log.cc:1196] T 1a5f291abcfd4c3daff56bbb4b268988 P 745d4b316d984514bf8648dc6f19369e: Deleting WAL directory at /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790205259943518-25487-0/minicluster-data/ts-1/wal/wals/1a5f291abcfd4c3daff56bbb4b268988
I20260923 23:16:43.154541 6275 ts_tablet_manager.cc:1939] T 1a5f291abcfd4c3daff56bbb4b268988 P 745d4b316d984514bf8648dc6f19369e: Deleting consensus metadata
I20260923 23:16:43.154907 5738 catalog_manager.cc:4841] TS 745d4b316d984514bf8648dc6f19369e (127.24.227.194:44393): tablet 1a5f291abcfd4c3daff56bbb4b268988 (table test-table [id=f8d3e62bde884e929da9f04a240f1ea1]) successfully deleted
W20260923 23:16:43.172092 5736 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.227.193:40957 (error 108)
I20260923 23:16:43.172984 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 5953
I20260923 23:16:43.180775 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 6086
I20260923 23:16:43.181393 6248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:43.180949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.193:40957 (local address 127.0.0.1:41388)
0923 23:16:43.181286 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0923 23:16:43.181345 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.193:40957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":317}
W20260923 23:16:43.181789 5736 catalog_manager.cc:4584] TS 49d3c58ecd3844a3a859080b3a3d2837 (127.24.227.193:40957): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1a5f291abcfd4c3daff56bbb4b268988: Network error: Client connection negotiation failed: client connection to 127.24.227.193:40957: connect: Connection refused (error 111)
I20260923 23:16:43.187369 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 5718
I20260923 23:16:43.199970 25487 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260923 23:16:43.533438 25487 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260923 23:16:43.533524 25487 mini_ranger.cc:66] Stopping Ranger...
I20260923 23:16:43.877307 25487 mini_ranger.cc:68] Stopped Ranger
2026-09-23 23:16:43.877 UTC [4785] LOG: received smart shutdown request
2026-09-23 23:16:43.881 UTC [4785] LOG: background worker "logical replication launcher" (PID 4794) exited with exit code 1
2026-09-23 23:16:43.881 UTC [4789] LOG: shutting down
2026-09-23 23:16:44.030 UTC [4785] LOG: database system is shut down
2026-09-23T23:16:44Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (45569 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790205259943518-25487-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 23:16:44 dist-test-slave-754n krb5kdc[6288](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 23:16:44 dist-test-slave-754n krb5kdc[6288](info): set up 2 sockets
Sep 23 23:16:44 dist-test-slave-754n krb5kdc[6288](info): commencing operation
krb5kdc: starting...
W20260923 23:16:45.151661 25487 mini_kdc.cc:121] Time spent starting KDC: real 0.994s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790205259943518-25487-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790205259943518-25487-0/krb5kdc/test-user.keytab.
Sep 23 23:16:45 dist-test-slave-754n krb5kdc[6288](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205405, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T23:16:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T23:16:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.227.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.227.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.227.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab.
Entry for principal kudu/127.24.227.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab.
I20260923 23:16:45.260463 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790205259943518-25487-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790205259943518-25487-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790205259943518-25487-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.254:39273
--webserver_interface=127.24.227.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab
--principal=kudu/127.24.227.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:40835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.227.254:39273,127.24.227.253:38449,127.24.227.252:45233
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:16:45.338013 6301 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:16:45.338265 6301 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:16:45.338286 6301 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:16:45.339404 6301 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 23:16:45.339443 6301 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:16:45.339457 6301 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:16:45.339502 6301 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 23:16:45.339526 6301 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 23:16:45.340981 6301 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:40835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790205259943518-25487-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790205259943518-25487-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.227.254:39273,127.24.227.253:38449,127.24.227.252:45233
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab
--principal=kudu/127.24.227.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.254:39273
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.227.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6301
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790205259943518-25487-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:16:45.341207 6301 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:16:45.341416 6301 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:16:45.345952 6309 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:16:45.345933 6307 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:16:45.345928 6306 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:16:45.345947 6301 server_base.cc:1034] running on GCE node
I20260923 23:16:45.346377 6301 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:16:45.346607 6301 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:16:45.347733 6301 hybrid_clock.cc:648] HybridClock initialized: now 1790205405347721 us; error 27 us; skew 500 ppm
Sep 23 23:16:45 dist-test-slave-754n krb5kdc[6288](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205405, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:16:45.349920 6301 init.cc:377] Logged in from keytab as kudu/127.24.227.254@KRBTEST.COM (short username kudu)
I20260923 23:16:45.350277 6301 webserver.cc:460] Webserver started at http://127.24.227.254:46517/ using document root <none> and password file <none>
I20260923 23:16:45.350433 6301 fs_manager.cc:359] Metadata directory not provided
I20260923 23:16:45.350468 6301 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:16:45.350544 6301 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:16:45.351372 6301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790205259943518-25487-0/minicluster-data/master-0/data/instance:
uuid: "7b15ad20e8b54b03aea340fd5795f929"
format_stamp: "Formatted at 2026-09-23 23:16:45 on dist-test-slave-754n"
server_key: "58296421824a0e97cd1a0199af5c940f"
server_key_iv: "f3edbaccf52aa09a47ccfb49ecfaab38"
server_key_version: "encryptionkey@0"
I20260923 23:16:45.351665 6301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790205259943518-25487-0/minicluster-data/master-0/wal/instance:
uuid: "7b15ad20e8b54b03aea340fd5795f929"
format_stamp: "Formatted at 2026-09-23 23:16:45 on dist-test-slave-754n"
server_key: "58296421824a0e97cd1a0199af5c940f"
server_key_iv: "f3edbaccf52aa09a47ccfb49ecfaab38"
server_key_version: "encryptionkey@0"
I20260923 23:16:45.352952 6301 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260923 23:16:45.353648 6316 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:16:45.353801 6301 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 23:16:45.353924 6301 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790205259943518-25487-0/minicluster-data/master-0/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790205259943518-25487-0/minicluster-data/master-0/wal
uuid: "7b15ad20e8b54b03aea340fd5795f929"
format_stamp: "Formatted at 2026-09-23 23:16:45 on dist-test-slave-754n"
server_key: "58296421824a0e97cd1a0199af5c940f"
server_key_iv: "f3edbaccf52aa09a47ccfb49ecfaab38"
server_key_version: "encryptionkey@0"
I20260923 23:16:45.353993 6301 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790205259943518-25487-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790205259943518-25487-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790205259943518-25487-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:16:45.363435 6301 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:16:45.365501 6301 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:16:45.365603 6301 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:16:45.369225 6301 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.254:39273
I20260923 23:16:45.369272 6368 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.254:39273 every 8 connection(s)
I20260923 23:16:45.369612 6301 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
I20260923 23:16:45.370383 6369 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:16:45.371737 6369 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.227.254" port: 39273 } has no permanent_uuid. Determining permanent_uuid...
I20260923 23:16:45.374045 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 6301
I20260923 23:16:45.374155 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790205259943518-25487-0/minicluster-data/master-0/wal/instance
I20260923 23:16:45.374327 25487 external_mini_cluster.cc:1428] Setting key 72034e0ba86024bde7302bb38576be25
Sep 23 23:16:45 dist-test-slave-754n krb5kdc[6288](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205405, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.254@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
I20260923 23:16:45.380996 6371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:45.372433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:39273 (local address 127.0.0.1:60208)
0923 23:16:45.372598 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0923 23:16:45.372630 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0923 23:16:45.373034 (+ 404us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:16:45.373268 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:16:45.373270 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:16:45.373475 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:16:45.373612 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:16:45.373616 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:16:45.374688 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:16:45.374689 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:16:45.375422 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:16:45.375425 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:16:45.375949 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:16:45.375950 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:16:45.376033 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:16:45.376449 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:16:45.376456 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:16:45.377732 (+ 1276us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:16:45.379677 (+ 1945us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:16:45.379683 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:16:45.379684 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:16:45.379860 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:16:45.380151 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:16:45.380151 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:16:45.380152 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:16:45.380223 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:16:45.380601 (+ 378us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:16:45.380604 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:16:45.380649 (+ 45us) client_negotiation.cc:769] Sending connection context
0923 23:16:45.380712 (+ 63us) client_negotiation.cc:240] Negotiation successful
0923 23:16:45.380812 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":84,"threads_started":1}
I20260923 23:16:45.381013 6372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:45.372597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60208 (local address 127.24.227.254:39273)
0923 23:16:45.372709 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0923 23:16:45.372713 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 23:16:45.372747 (+ 34us) server_negotiation.cc:407] Connection header received
0923 23:16:45.373157 (+ 410us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:16:45.373159 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:16:45.373174 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:16:45.373244 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:16:45.373681 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:45.374602 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:45.375517 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:45.375826 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:45.375963 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:16:45.377807 (+ 1844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:16:45.377813 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:16:45.377814 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:16:45.377821 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:16:45.379597 (+ 1776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:16:45.379976 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:16:45.379980 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:16:45.379980 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:16:45.380088 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:16:45.380281 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:16:45.380282 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:16:45.380282 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:16:45.380537 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:16:45.380617 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0923 23:16:45.380765 (+ 148us) server_negotiation.cc:299] Negotiation successful
0923 23:16:45.380886 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":30,"server-negotiator.queue_time_us":84,"thread_start_us":60,"threads_started":1}
WARNING: no policy specified for kudu/127.24.227.253@KRBTEST.COM; defaulting to no policy
I20260923 23:16:45.381687 6369 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.227.253" port: 38449 } has no permanent_uuid. Determining permanent_uuid...
I20260923 23:16:45.381974 6371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:45.381863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.253:38449 (local address 127.0.0.1:39122)
0923 23:16:45.381908 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0923 23:16:45.381951 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.253:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260923 23:16:45.382107 6320 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.227.253:38449: connect: Connection refused (error 111)
I20260923 23:16:45.382504 6371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:45.382416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.253:38449 (local address 127.0.0.1:39138)
0923 23:16:45.382451 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0923 23:16:45.382474 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.253:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260923 23:16:45.382614 6369 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.227.253:38449: Network error: Client connection negotiation failed: client connection to 127.24.227.253:38449: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.227.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.227.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.253.keytab.
Entry for principal kudu/127.24.227.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.253.keytab.
I20260923 23:16:45.398295 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790205259943518-25487-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790205259943518-25487-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790205259943518-25487-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790205259943518-25487-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.253:38449
--webserver_interface=127.24.227.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.253.keytab
--principal=kudu/127.24.227.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:40835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.227.254:39273,127.24.227.253:38449,127.24.227.252:45233
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 23:16:45.431803 6369 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.227.253" port: 38449 } attempt: 1
I20260923 23:16:45.432287 6371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:45.432146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.253:38449 (local address 127.0.0.1:39142)
0923 23:16:45.432210 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0923 23:16:45.432253 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.253:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260923 23:16:45.432622 6371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:45.432529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.253:38449 (local address 127.0.0.1:39152)
0923 23:16:45.432573 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0923 23:16:45.432601 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.253:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260923 23:16:45.432740 6369 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.227.253:38449: Network error: Client connection negotiation failed: client connection to 127.24.227.253:38449: connect: Connection refused (error 111)
W20260923 23:16:45.473155 6376 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:16:45.473346 6376 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:16:45.473367 6376 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:16:45.474499 6376 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 23:16:45.474538 6376 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:16:45.474552 6376 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:16:45.474601 6376 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 23:16:45.474625 6376 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 23:16:45.476089 6376 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:40835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790205259943518-25487-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790205259943518-25487-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.24.227.254:39273,127.24.227.253:38449,127.24.227.252:45233
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.253.keytab
--principal=kudu/127.24.227.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.253:38449
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790205259943518-25487-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.24.227.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6376
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790205259943518-25487-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:16:45.476310 6376 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:16:45.476513 6376 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:16:45.481611 6384 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:16:45.481808 6376 server_base.cc:1034] running on GCE node
W20260923 23:16:45.481877 6382 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:16:45.481911 6381 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:16:45.482183 6376 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:16:45.482494 6376 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:16:45.483646 6376 hybrid_clock.cc:648] HybridClock initialized: now 1790205405483621 us; error 38 us; skew 500 ppm
Sep 23 23:16:45 dist-test-slave-754n krb5kdc[6288](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205405, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:16:45.486663 6376 init.cc:377] Logged in from keytab as kudu/127.24.227.253@KRBTEST.COM (short username kudu)
I20260923 23:16:45.487169 6376 webserver.cc:460] Webserver started at http://127.24.227.253:37893/ using document root <none> and password file <none>
I20260923 23:16:45.487425 6376 fs_manager.cc:359] Metadata directory not provided
I20260923 23:16:45.487490 6376 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:16:45.487623 6376 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:16:45.488580 6376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790205259943518-25487-0/minicluster-data/master-1/data/instance:
uuid: "3d13802e5bd44aa4a9979c188ca543cd"
format_stamp: "Formatted at 2026-09-23 23:16:45 on dist-test-slave-754n"
server_key: "74c09fc7a79b7ae9c2ddba509d882a4e"
server_key_iv: "3812a7a866e015ee6707d44788b437d4"
server_key_version: "encryptionkey@0"
I20260923 23:16:45.488850 6376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790205259943518-25487-0/minicluster-data/master-1/wal/instance:
uuid: "3d13802e5bd44aa4a9979c188ca543cd"
format_stamp: "Formatted at 2026-09-23 23:16:45 on dist-test-slave-754n"
server_key: "74c09fc7a79b7ae9c2ddba509d882a4e"
server_key_iv: "3812a7a866e015ee6707d44788b437d4"
server_key_version: "encryptionkey@0"
I20260923 23:16:45.490020 6376 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260923 23:16:45.490612 6391 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:16:45.490767 6376 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 23:16:45.490839 6376 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790205259943518-25487-0/minicluster-data/master-1/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790205259943518-25487-0/minicluster-data/master-1/wal
uuid: "3d13802e5bd44aa4a9979c188ca543cd"
format_stamp: "Formatted at 2026-09-23 23:16:45 on dist-test-slave-754n"
server_key: "74c09fc7a79b7ae9c2ddba509d882a4e"
server_key_iv: "3812a7a866e015ee6707d44788b437d4"
server_key_version: "encryptionkey@0"
I20260923 23:16:45.490916 6376 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790205259943518-25487-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790205259943518-25487-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790205259943518-25487-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:16:45.500891 6369 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.227.253" port: 38449 } attempt: 2
I20260923 23:16:45.501226 6371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:45.501123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.253:38449 (local address 127.0.0.1:39156)
0923 23:16:45.501176 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0923 23:16:45.501205 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.253:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260923 23:16:45.501513 6371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:45.501430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.253:38449 (local address 127.0.0.1:39164)
0923 23:16:45.501469 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0923 23:16:45.501492 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.253:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260923 23:16:45.501593 6369 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.227.253:38449: Network error: Client connection negotiation failed: client connection to 127.24.227.253:38449: connect: Connection refused (error 111)
I20260923 23:16:45.502955 6376 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:16:45.504637 6376 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:16:45.504776 6376 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:16:45.508504 6376 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.253:38449
I20260923 23:16:45.508556 6443 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.253:38449 every 8 connection(s)
I20260923 23:16:45.508874 6376 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790205259943518-25487-0/minicluster-data/master-1/data/info.pb
I20260923 23:16:45.509648 6444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:16:45.511008 6444 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.227.254" port: 39273 } has no permanent_uuid. Determining permanent_uuid...
I20260923 23:16:45.512207 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 6376
I20260923 23:16:45.512320 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790205259943518-25487-0/minicluster-data/master-1/wal/instance
I20260923 23:16:45.512513 25487 external_mini_cluster.cc:1428] Setting key 5eeab5ed8db150c3e8f7907ab7a20064
Sep 23 23:16:45 dist-test-slave-754n krb5kdc[6288](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205405, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.253@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.24.227.252@KRBTEST.COM; defaulting to no policy
I20260923 23:16:45.520782 6372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:45.511594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60210 (local address 127.24.227.254:39273)
0923 23:16:45.511664 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0923 23:16:45.511667 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 23:16:45.511825 (+ 158us) server_negotiation.cc:407] Connection header received
0923 23:16:45.512354 (+ 529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:16:45.512355 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:16:45.512370 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:16:45.512433 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:16:45.513059 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:45.513682 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:45.515110 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:45.515360 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:45.515450 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:16:45.517899 (+ 2449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:16:45.517901 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:16:45.517901 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:16:45.517908 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:16:45.519312 (+ 1404us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:16:45.519803 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:16:45.519804 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:16:45.519804 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:16:45.519853 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:16:45.520181 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:16:45.520181 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:16:45.520182 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:16:45.520281 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:16:45.520327 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0923 23:16:45.520665 (+ 338us) server_negotiation.cc:299] Negotiation successful
0923 23:16:45.520727 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260923 23:16:45.521016 6446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:45.511567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:39273 (local address 127.0.0.1:60210)
0923 23:16:45.511717 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0923 23:16:45.511748 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0923 23:16:45.512247 (+ 499us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:16:45.512502 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:16:45.512505 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:16:45.512779 (+ 274us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:16:45.512970 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:16:45.512974 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:16:45.513791 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:16:45.513792 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:16:45.515016 (+ 1224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:16:45.515019 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:16:45.515460 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:16:45.515461 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:16:45.515584 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:16:45.516103 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:16:45.516111 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:16:45.517803 (+ 1692us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:16:45.519407 (+ 1604us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:16:45.519414 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:16:45.519417 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:16:45.519724 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:16:45.519971 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:16:45.519971 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:16:45.519972 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:16:45.520109 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:16:45.520340 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:16:45.520346 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:16:45.520527 (+ 181us) client_negotiation.cc:769] Sending connection context
0923 23:16:45.520652 (+ 125us) client_negotiation.cc:240] Negotiation successful
0923 23:16:45.520798 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":82,"threads_started":1}
I20260923 23:16:45.521550 6444 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.227.253" port: 38449 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.227.252@KRBTEST.COM" created.
Sep 23 23:16:45 dist-test-slave-754n krb5kdc[6288](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205405, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.253@KRBTEST.COM for kudu/127.24.227.253@KRBTEST.COM
I20260923 23:16:45.529223 6446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:45.521790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.253:38449 (local address 127.0.0.1:39178)
0923 23:16:45.521836 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0923 23:16:45.521845 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 23:16:45.521903 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:16:45.522155 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:16:45.522156 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:16:45.522260 (+ 104us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:16:45.522304 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:16:45.522305 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:16:45.523233 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:16:45.523234 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:16:45.524089 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:16:45.524092 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:16:45.524699 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:16:45.524700 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:16:45.524747 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:16:45.525166 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:16:45.525171 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:16:45.526522 (+ 1351us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:16:45.528173 (+ 1651us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:16:45.528174 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:16:45.528174 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:16:45.528397 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:16:45.528578 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:16:45.528579 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:16:45.528580 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:16:45.528644 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:16:45.528877 (+ 233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:16:45.528878 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:16:45.528923 (+ 45us) client_negotiation.cc:769] Sending connection context
0923 23:16:45.528995 (+ 72us) client_negotiation.cc:240] Negotiation successful
0923 23:16:45.529093 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260923 23:16:45.529320 6448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:45.521856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39178 (local address 127.24.227.253:38449)
0923 23:16:45.522003 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0923 23:16:45.522007 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 23:16:45.522025 (+ 18us) server_negotiation.cc:407] Connection header received
0923 23:16:45.522045 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:16:45.522052 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:16:45.522062 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:16:45.522154 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:16:45.522353 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:45.523150 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:45.524171 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:45.524624 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:45.524703 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:16:45.526602 (+ 1899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:16:45.526608 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:16:45.526608 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:16:45.526614 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:16:45.528099 (+ 1485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:16:45.528469 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:16:45.528471 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:16:45.528471 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:16:45.528517 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:16:45.528707 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:16:45.528707 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:16:45.528707 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:16:45.528817 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:16:45.528870 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0923 23:16:45.529027 (+ 157us) server_negotiation.cc:299] Negotiation successful
0923 23:16:45.529164 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":55,"server-negotiator.queue_time_us":121,"thread_start_us":98,"threads_started":1}
I20260923 23:16:45.529881 6444 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.227.252" port: 45233 } has no permanent_uuid. Determining permanent_uuid...
I20260923 23:16:45.530164 6446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:45.530066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.252:45233 (local address 127.0.0.1:59658)
0923 23:16:45.530097 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0923 23:16:45.530140 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.252:45233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
W20260923 23:16:45.530296 6393 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.227.252:45233: connect: Connection refused (error 111)
I20260923 23:16:45.530707 6446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:45.530633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.252:45233 (local address 127.0.0.1:59670)
0923 23:16:45.530670 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0923 23:16:45.530692 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.252:45233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260923 23:16:45.530833 6444 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.227.252:45233: Network error: Client connection negotiation failed: client connection to 127.24.227.252:45233: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.227.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.252.keytab.
Entry for principal kudu/127.24.227.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.252.keytab.
I20260923 23:16:45.536580 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790205259943518-25487-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790205259943518-25487-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790205259943518-25487-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790205259943518-25487-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.252:45233
--webserver_interface=127.24.227.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.252.keytab
--principal=kudu/127.24.227.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:40835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.227.254:39273,127.24.227.253:38449,127.24.227.252:45233
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 23:16:45.580072 6444 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.227.252" port: 45233 } attempt: 1
I20260923 23:16:45.580495 6446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:45.580341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.252:45233 (local address 127.0.0.1:59672)
0923 23:16:45.580422 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0923 23:16:45.580463 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.252:45233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260923 23:16:45.580821 6446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:45.580741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.252:45233 (local address 127.0.0.1:59688)
0923 23:16:45.580778 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0923 23:16:45.580804 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.252:45233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260923 23:16:45.580948 6444 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.227.252:45233: Network error: Client connection negotiation failed: client connection to 127.24.227.252:45233: connect: Connection refused (error 111)
I20260923 23:16:45.592726 6369 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.227.253" port: 38449 } attempt: 3
Sep 23 23:16:45 dist-test-slave-754n krb5kdc[6288](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205405, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.254@KRBTEST.COM for kudu/127.24.227.253@KRBTEST.COM
I20260923 23:16:45.599344 6448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:45.593057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39190 (local address 127.24.227.253:38449)
0923 23:16:45.593116 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0923 23:16:45.593118 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:16:45.593130 (+ 12us) server_negotiation.cc:407] Connection header received
0923 23:16:45.593210 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:16:45.593211 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:16:45.593223 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:16:45.593292 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:16:45.593573 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:45.594042 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:45.594776 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:45.595008 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:45.595081 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:16:45.597185 (+ 2104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:16:45.597186 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:16:45.597187 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:16:45.597192 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:16:45.598388 (+ 1196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:16:45.598733 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:16:45.598733 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:16:45.598734 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:16:45.598777 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:16:45.598952 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:16:45.598953 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:16:45.598953 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:16:45.599038 (+ 85us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:16:45.599081 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0923 23:16:45.599243 (+ 162us) server_negotiation.cc:299] Negotiation successful
0923 23:16:45.599295 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260923 23:16:45.599418 6371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:45.593028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.253:38449 (local address 127.0.0.1:39190)
0923 23:16:45.593088 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0923 23:16:45.593097 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 23:16:45.593143 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:16:45.593298 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:16:45.593299 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:16:45.593445 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:16:45.593501 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:16:45.593503 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:16:45.594151 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:16:45.594152 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:16:45.594671 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:16:45.594672 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:16:45.595077 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:16:45.595078 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:16:45.595153 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:16:45.595626 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:16:45.595630 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:16:45.597106 (+ 1476us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:16:45.598463 (+ 1357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:16:45.598464 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:16:45.598464 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:16:45.598666 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:16:45.598832 (+ 166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:16:45.598833 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:16:45.598833 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:16:45.598896 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:16:45.599096 (+ 200us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:16:45.599098 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:16:45.599152 (+ 54us) client_negotiation.cc:769] Sending connection context
0923 23:16:45.599228 (+ 76us) client_negotiation.cc:240] Negotiation successful
0923 23:16:45.599325 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260923 23:16:45.599795 6369 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.227.252" port: 45233 } has no permanent_uuid. Determining permanent_uuid...
I20260923 23:16:45.600102 6371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:45.600018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.252:45233 (local address 127.0.0.1:59694)
0923 23:16:45.600058 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0923 23:16:45.600084 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.252:45233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260923 23:16:45.600394 6371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:45.600307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.252:45233 (local address 127.0.0.1:59700)
0923 23:16:45.600344 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0923 23:16:45.600366 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.252:45233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260923 23:16:45.600476 6369 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.227.252:45233: Network error: Client connection negotiation failed: client connection to 127.24.227.252:45233: connect: Connection refused (error 111)
W20260923 23:16:45.612834 6451 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:16:45.613049 6451 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:16:45.613114 6451 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:16:45.614202 6451 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 23:16:45.614259 6451 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:16:45.614284 6451 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:16:45.614310 6451 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 23:16:45.614338 6451 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 23:16:45.615759 6451 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:40835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790205259943518-25487-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790205259943518-25487-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.24.227.254:39273,127.24.227.253:38449,127.24.227.252:45233
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.252.keytab
--principal=kudu/127.24.227.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.252:45233
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790205259943518-25487-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.24.227.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6451
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790205259943518-25487-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:16:45.616042 6451 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:16:45.616261 6451 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260923 23:16:45.620532 6451 server_base.cc:1034] running on GCE node
W20260923 23:16:45.620500 6459 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:16:45.620486 6457 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:16:45.620509 6456 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:16:45.620931 6451 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:16:45.621188 6451 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:16:45.622315 6451 hybrid_clock.cc:648] HybridClock initialized: now 1790205405622295 us; error 34 us; skew 500 ppm
Sep 23 23:16:45 dist-test-slave-754n krb5kdc[6288](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205405, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:16:45.624442 6451 init.cc:377] Logged in from keytab as kudu/127.24.227.252@KRBTEST.COM (short username kudu)
I20260923 23:16:45.624836 6451 webserver.cc:460] Webserver started at http://127.24.227.252:34697/ using document root <none> and password file <none>
I20260923 23:16:45.625025 6451 fs_manager.cc:359] Metadata directory not provided
I20260923 23:16:45.625064 6451 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:16:45.625145 6451 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:16:45.626032 6451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790205259943518-25487-0/minicluster-data/master-2/data/instance:
uuid: "f08b4fea623e461ebeb824f70032555c"
format_stamp: "Formatted at 2026-09-23 23:16:45 on dist-test-slave-754n"
server_key: "406711bcd915c61577c1f36ed6158f1c"
server_key_iv: "6fb99a82684c3cc81fa2f6756113234c"
server_key_version: "encryptionkey@0"
I20260923 23:16:45.626312 6451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790205259943518-25487-0/minicluster-data/master-2/wal/instance:
uuid: "f08b4fea623e461ebeb824f70032555c"
format_stamp: "Formatted at 2026-09-23 23:16:45 on dist-test-slave-754n"
server_key: "406711bcd915c61577c1f36ed6158f1c"
server_key_iv: "6fb99a82684c3cc81fa2f6756113234c"
server_key_version: "encryptionkey@0"
I20260923 23:16:45.627437 6451 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 23:16:45.628057 6466 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:16:45.628199 6451 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 23:16:45.628283 6451 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790205259943518-25487-0/minicluster-data/master-2/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790205259943518-25487-0/minicluster-data/master-2/wal
uuid: "f08b4fea623e461ebeb824f70032555c"
format_stamp: "Formatted at 2026-09-23 23:16:45 on dist-test-slave-754n"
server_key: "406711bcd915c61577c1f36ed6158f1c"
server_key_iv: "6fb99a82684c3cc81fa2f6756113234c"
server_key_version: "encryptionkey@0"
I20260923 23:16:45.628350 6451 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790205259943518-25487-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790205259943518-25487-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790205259943518-25487-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:16:45.631592 6369 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.227.252" port: 45233 } attempt: 1
I20260923 23:16:45.631937 6371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:45.631782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.252:45233 (local address 127.0.0.1:59714)
0923 23:16:45.631832 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0923 23:16:45.631856 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.252:45233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260923 23:16:45.632205 6371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:45.632125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.252:45233 (local address 127.0.0.1:59716)
0923 23:16:45.632164 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0923 23:16:45.632186 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.252:45233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260923 23:16:45.632290 6369 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.227.252:45233: Network error: Client connection negotiation failed: client connection to 127.24.227.252:45233: connect: Connection refused (error 111)
I20260923 23:16:45.635723 6451 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:16:45.637712 6451 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:16:45.637840 6451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:16:45.641368 6451 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.252:45233
I20260923 23:16:45.641398 6518 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.252:45233 every 8 connection(s)
I20260923 23:16:45.641680 6451 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790205259943518-25487-0/minicluster-data/master-2/data/info.pb
I20260923 23:16:45.642409 6519 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:16:45.643630 6519 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.227.254" port: 39273 } has no permanent_uuid. Determining permanent_uuid...
Sep 23 23:16:45 dist-test-slave-754n krb5kdc[6288](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205405, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.252@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
I20260923 23:16:45.649094 6444 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.227.252" port: 45233 } attempt: 2
I20260923 23:16:45.650449 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 6451
I20260923 23:16:45.650568 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790205259943518-25487-0/minicluster-data/master-2/wal/instance
I20260923 23:16:45.650774 25487 external_mini_cluster.cc:1428] Setting key 6a4d3b96f33fec3f5debd944fc3fa536
I20260923 23:16:45.651518 6372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:45.644237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60212 (local address 127.24.227.254:39273)
0923 23:16:45.644299 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0923 23:16:45.644301 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:16:45.644456 (+ 155us) server_negotiation.cc:407] Connection header received
0923 23:16:45.644793 (+ 337us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:16:45.644794 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:16:45.644802 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:16:45.644846 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:16:45.645394 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:45.645829 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:45.646680 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:45.646839 (+ 159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:45.646886 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:16:45.648612 (+ 1726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:16:45.648613 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:16:45.648614 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:16:45.648619 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:16:45.649911 (+ 1292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:16:45.650337 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:16:45.650338 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:16:45.650338 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:16:45.650406 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:16:45.650712 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:16:45.650713 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:16:45.650713 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:16:45.650842 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:16:45.650910 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0923 23:16:45.651357 (+ 447us) server_negotiation.cc:299] Negotiation successful
0923 23:16:45.651448 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260923 23:16:45.651684 6521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:45.644202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:39273 (local address 127.0.0.1:60212)
0923 23:16:45.644360 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0923 23:16:45.644391 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0923 23:16:45.644730 (+ 339us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:16:45.644899 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:16:45.644904 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:16:45.645155 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:16:45.645326 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:16:45.645330 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:16:45.645909 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:16:45.645909 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:16:45.646619 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:16:45.646620 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:16:45.646908 (+ 288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:16:45.646909 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:16:45.647014 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:16:45.647356 (+ 342us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:16:45.647363 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:16:45.648538 (+ 1175us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:16:45.649974 (+ 1436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:16:45.649978 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:16:45.649980 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:16:45.650240 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:16:45.650479 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:16:45.650479 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:16:45.650480 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:16:45.650596 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:16:45.650900 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:16:45.650904 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:16:45.651176 (+ 272us) client_negotiation.cc:769] Sending connection context
0923 23:16:45.651310 (+ 134us) client_negotiation.cc:240] Negotiation successful
0923 23:16:45.651465 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":83,"threads_started":1}
I20260923 23:16:45.652319 6519 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.227.253" port: 38449 } has no permanent_uuid. Determining permanent_uuid...
Sep 23 23:16:45 dist-test-slave-754n krb5kdc[6288](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205405, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.253@KRBTEST.COM for kudu/127.24.227.252@KRBTEST.COM
Sep 23 23:16:45 dist-test-slave-754n krb5kdc[6288](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205405, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.252@KRBTEST.COM for kudu/127.24.227.253@KRBTEST.COM
Sep 23 23:16:45 dist-test-slave-754n krb5kdc[6288](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205405, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.227.252@KRBTEST.COM
I20260923 23:16:45.657220 6446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:45.649324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.252:45233 (local address 127.0.0.1:59720)
0923 23:16:45.649379 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0923 23:16:45.649388 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 23:16:45.649423 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:16:45.649668 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:16:45.649669 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:16:45.649802 (+ 133us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:16:45.649835 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:16:45.649837 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:16:45.650986 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:16:45.650987 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:16:45.651688 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:16:45.651691 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:16:45.652372 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:16:45.652373 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:16:45.652434 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:16:45.652815 (+ 381us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:16:45.652819 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:16:45.654038 (+ 1219us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:16:45.656145 (+ 2107us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:16:45.656146 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:16:45.656147 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:16:45.656358 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:16:45.656561 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:16:45.656562 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:16:45.656563 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:16:45.656624 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:16:45.656951 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:16:45.656952 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:16:45.657011 (+ 59us) client_negotiation.cc:769] Sending connection context
0923 23:16:45.657085 (+ 74us) client_negotiation.cc:240] Negotiation successful
0923 23:16:45.657150 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260923 23:16:45.657302 6522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:45.649399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59720 (local address 127.24.227.252:45233)
0923 23:16:45.649537 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0923 23:16:45.649540 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 23:16:45.649555 (+ 15us) server_negotiation.cc:407] Connection header received
0923 23:16:45.649571 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:16:45.649572 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:16:45.649582 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:16:45.649653 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:16:45.649919 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:45.650917 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:45.651770 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:45.652295 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:45.652375 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:16:45.654117 (+ 1742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:16:45.654123 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:16:45.654123 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:16:45.654130 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:16:45.656058 (+ 1928us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:16:45.656446 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:16:45.656448 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:16:45.656449 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:16:45.656500 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:16:45.656702 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:16:45.656704 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:16:45.656704 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:16:45.656859 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:16:45.656951 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0923 23:16:45.657134 (+ 183us) server_negotiation.cc:299] Negotiation successful
0923 23:16:45.657202 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":66,"threads_started":1}
I20260923 23:16:45.658766 6444 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d13802e5bd44aa4a9979c188ca543cd: Bootstrap starting.
I20260923 23:16:45.659641 6444 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3d13802e5bd44aa4a9979c188ca543cd: Neither blocks nor log segments found. Creating new log.
I20260923 23:16:45.659996 6444 log.cc:824] T 00000000000000000000000000000000 P 3d13802e5bd44aa4a9979c188ca543cd: Log is configured to *not* fsync() on all Append() calls
I20260923 23:16:45.660872 6523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:45.651063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59732 (local address 127.24.227.252:45233)
0923 23:16:45.651178 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0923 23:16:45.651180 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:16:45.651633 (+ 453us) server_negotiation.cc:407] Connection header received
0923 23:16:45.651653 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:16:45.651654 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:16:45.651663 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:16:45.651735 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:16:45.652636 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:45.653357 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:45.654309 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:45.654723 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:45.654829 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:16:45.657960 (+ 3131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:16:45.657961 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:16:45.657961 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:16:45.657967 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:16:45.659709 (+ 1742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:16:45.660104 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:16:45.660105 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:16:45.660105 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:16:45.660157 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:16:45.660362 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:16:45.660364 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:16:45.660364 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:16:45.660499 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:16:45.660639 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0923 23:16:45.660751 (+ 112us) server_negotiation.cc:299] Negotiation successful
0923 23:16:45.660811 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":79,"threads_started":1}
I20260923 23:16:45.661300 6444 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d13802e5bd44aa4a9979c188ca543cd: No bootstrap required, opened a new log
I20260923 23:16:45.662174 6521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:45.652564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.253:38449 (local address 127.0.0.1:39192)
0923 23:16:45.652826 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0923 23:16:45.652836 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 23:16:45.652885 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:16:45.652999 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:16:45.653000 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:16:45.653144 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:16:45.653188 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:16:45.653189 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:16:45.654001 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:16:45.654002 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:16:45.654698 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:16:45.654700 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:16:45.655149 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:16:45.655150 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:16:45.655223 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:16:45.655547 (+ 324us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:16:45.655552 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:16:45.657456 (+ 1904us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:16:45.661220 (+ 3764us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:16:45.661220 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:16:45.661221 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:16:45.661447 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:16:45.661660 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:16:45.661660 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:16:45.661661 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:16:45.661719 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:16:45.661997 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:16:45.661997 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:16:45.662025 (+ 28us) client_negotiation.cc:769] Sending connection context
0923 23:16:45.662069 (+ 44us) client_negotiation.cc:240] Negotiation successful
0923 23:16:45.662125 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247}
I20260923 23:16:45.662241 6448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:45.652641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39192 (local address 127.24.227.253:38449)
0923 23:16:45.652674 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0923 23:16:45.652676 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:16:45.652891 (+ 215us) server_negotiation.cc:407] Connection header received
0923 23:16:45.652936 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:16:45.652937 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:16:45.652947 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:16:45.652998 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:16:45.653257 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:45.653921 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:45.654769 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:45.655085 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:45.655148 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:16:45.657526 (+ 2378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:16:45.657527 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:16:45.657527 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:16:45.657533 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:16:45.661144 (+ 3611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:16:45.661529 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:16:45.661531 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:16:45.661531 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:16:45.661585 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:16:45.661778 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:16:45.661779 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:16:45.661779 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:16:45.661929 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:16:45.662007 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0923 23:16:45.662092 (+ 85us) server_negotiation.cc:299] Negotiation successful
0923 23:16:45.662159 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260923 23:16:45.662560 6519 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.227.252" port: 45233 } has no permanent_uuid. Determining permanent_uuid...
I20260923 23:16:45.663239 6444 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3d13802e5bd44aa4a9979c188ca543cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b15ad20e8b54b03aea340fd5795f929" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 39273 } } peers { permanent_uuid: "3d13802e5bd44aa4a9979c188ca543cd" member_type: VOTER last_known_addr { host: "127.24.227.253" port: 38449 } } peers { permanent_uuid: "f08b4fea623e461ebeb824f70032555c" member_type: VOTER last_known_addr { host: "127.24.227.252" port: 45233 } }
I20260923 23:16:45.663331 6444 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3d13802e5bd44aa4a9979c188ca543cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:16:45.663352 6444 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3d13802e5bd44aa4a9979c188ca543cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d13802e5bd44aa4a9979c188ca543cd, State: Initialized, Role: FOLLOWER
I20260923 23:16:45.663477 6444 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3d13802e5bd44aa4a9979c188ca543cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b15ad20e8b54b03aea340fd5795f929" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 39273 } } peers { permanent_uuid: "3d13802e5bd44aa4a9979c188ca543cd" member_type: VOTER last_known_addr { host: "127.24.227.253" port: 38449 } } peers { permanent_uuid: "f08b4fea623e461ebeb824f70032555c" member_type: VOTER last_known_addr { host: "127.24.227.252" port: 45233 } }
I20260923 23:16:45.663749 6444 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3d13802e5bd44aa4a9979c188ca543cd [sys.catalog]: configured and running, proceeding with master startup.
I20260923 23:16:45.663727 6526 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d13802e5bd44aa4a9979c188ca543cd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b15ad20e8b54b03aea340fd5795f929" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 39273 } } peers { permanent_uuid: "3d13802e5bd44aa4a9979c188ca543cd" member_type: VOTER last_known_addr { host: "127.24.227.253" port: 38449 } } peers { permanent_uuid: "f08b4fea623e461ebeb824f70032555c" member_type: VOTER last_known_addr { host: "127.24.227.252" port: 45233 } } }
I20260923 23:16:45.663823 6526 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d13802e5bd44aa4a9979c188ca543cd [sys.catalog]: This master's current role is: FOLLOWER
W20260923 23:16:45.665683 6537 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3d13802e5bd44aa4a9979c188ca543cd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 23:16:45.665781 6537 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Sep 23 23:16:45 dist-test-slave-754n krb5kdc[6288](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205405, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.252@KRBTEST.COM for kudu/127.24.227.252@KRBTEST.COM
I20260923 23:16:45.665856 6376 master_runner.cc:186] Error getting master registration for 127.24.227.254:39273: OK, instance_id { permanent_uuid: "7b15ad20e8b54b03aea340fd5795f929" instance_seqno: 1790205405368225 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260923 23:16:45.668438 6376 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Sep 23 23:16:45 dist-test-slave-754n krb5kdc[6288](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205405, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
I20260923 23:16:45.669636 6521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:45.662764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.252:45233 (local address 127.0.0.1:59734)
0923 23:16:45.662804 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0923 23:16:45.662815 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 23:16:45.662863 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:16:45.662974 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:16:45.662974 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:16:45.663094 (+ 120us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:16:45.663116 (+ 22us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:16:45.663117 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:16:45.663969 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:16:45.663970 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:16:45.664557 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:16:45.664559 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:16:45.664976 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:16:45.664977 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:16:45.665027 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:16:45.665395 (+ 368us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:16:45.665400 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:16:45.666667 (+ 1267us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:16:45.668693 (+ 2026us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:16:45.668694 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:16:45.668695 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:16:45.668892 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:16:45.669085 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:16:45.669086 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:16:45.669086 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:16:45.669149 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:16:45.669434 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:16:45.669434 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:16:45.669467 (+ 33us) client_negotiation.cc:769] Sending connection context
0923 23:16:45.669521 (+ 54us) client_negotiation.cc:240] Negotiation successful
0923 23:16:45.669589 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260923 23:16:45.669839 6523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:45.662795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59734 (local address 127.24.227.252:45233)
0923 23:16:45.662825 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0923 23:16:45.662827 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:16:45.662850 (+ 23us) server_negotiation.cc:407] Connection header received
0923 23:16:45.662916 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:16:45.662917 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:16:45.662925 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:16:45.662963 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:16:45.663194 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:45.663852 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:45.664642 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:45.664878 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:45.664976 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:16:45.666744 (+ 1768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:16:45.666745 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:16:45.666746 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:16:45.666752 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:16:45.668579 (+ 1827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:16:45.668964 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:16:45.668965 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:16:45.668965 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:16:45.669018 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:16:45.669215 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:16:45.669216 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:16:45.669216 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:16:45.669357 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:16:45.669413 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0923 23:16:45.669742 (+ 329us) server_negotiation.cc:299] Negotiation successful
0923 23:16:45.669802 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260923 23:16:45.670657 6519 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f08b4fea623e461ebeb824f70032555c: Bootstrap starting.
I20260923 23:16:45.671168 6519 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f08b4fea623e461ebeb824f70032555c: Neither blocks nor log segments found. Creating new log.
I20260923 23:16:45.671479 6519 log.cc:824] T 00000000000000000000000000000000 P f08b4fea623e461ebeb824f70032555c: Log is configured to *not* fsync() on all Append() calls
I20260923 23:16:45.672437 6519 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f08b4fea623e461ebeb824f70032555c: No bootstrap required, opened a new log
I20260923 23:16:45.673359 6372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:45.661666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60214 (local address 127.24.227.254:39273)
0923 23:16:45.661703 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0923 23:16:45.661705 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:16:45.665041 (+ 3336us) server_negotiation.cc:407] Connection header received
0923 23:16:45.665099 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:16:45.665100 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:16:45.665111 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:16:45.665162 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:16:45.665774 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:45.666490 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:45.667189 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:45.667395 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:45.667472 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:16:45.669541 (+ 2069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:16:45.669542 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:16:45.669543 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:16:45.669549 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:16:45.671436 (+ 1887us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:16:45.672556 (+ 1120us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:16:45.672558 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:16:45.672558 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:16:45.672621 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:16:45.672857 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:16:45.672859 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:16:45.672859 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:16:45.672988 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:16:45.673161 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0923 23:16:45.673217 (+ 56us) server_negotiation.cc:299] Negotiation successful
0923 23:16:45.673287 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260923 23:16:45.674270 6519 raft_consensus.cc:348] T 00000000000000000000000000000000 P f08b4fea623e461ebeb824f70032555c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b15ad20e8b54b03aea340fd5795f929" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 39273 } } peers { permanent_uuid: "3d13802e5bd44aa4a9979c188ca543cd" member_type: VOTER last_known_addr { host: "127.24.227.253" port: 38449 } } peers { permanent_uuid: "f08b4fea623e461ebeb824f70032555c" member_type: VOTER last_known_addr { host: "127.24.227.252" port: 45233 } }
I20260923 23:16:45.674404 6519 raft_consensus.cc:374] T 00000000000000000000000000000000 P f08b4fea623e461ebeb824f70032555c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:16:45.674449 6519 raft_consensus.cc:729] T 00000000000000000000000000000000 P f08b4fea623e461ebeb824f70032555c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f08b4fea623e461ebeb824f70032555c, State: Initialized, Role: FOLLOWER
I20260923 23:16:45.674563 6519 consensus_queue.cc:260] T 00000000000000000000000000000000 P f08b4fea623e461ebeb824f70032555c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b15ad20e8b54b03aea340fd5795f929" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 39273 } } peers { permanent_uuid: "3d13802e5bd44aa4a9979c188ca543cd" member_type: VOTER last_known_addr { host: "127.24.227.253" port: 38449 } } peers { permanent_uuid: "f08b4fea623e461ebeb824f70032555c" member_type: VOTER last_known_addr { host: "127.24.227.252" port: 45233 } }
I20260923 23:16:45.674834 6519 sys_catalog.cc:564] T 00000000000000000000000000000000 P f08b4fea623e461ebeb824f70032555c [sys.catalog]: configured and running, proceeding with master startup.
I20260923 23:16:45.674794 6540 sys_catalog.cc:455] T 00000000000000000000000000000000 P f08b4fea623e461ebeb824f70032555c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b15ad20e8b54b03aea340fd5795f929" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 39273 } } peers { permanent_uuid: "3d13802e5bd44aa4a9979c188ca543cd" member_type: VOTER last_known_addr { host: "127.24.227.253" port: 38449 } } peers { permanent_uuid: "f08b4fea623e461ebeb824f70032555c" member_type: VOTER last_known_addr { host: "127.24.227.252" port: 45233 } } }
I20260923 23:16:45.675009 6540 sys_catalog.cc:458] T 00000000000000000000000000000000 P f08b4fea623e461ebeb824f70032555c [sys.catalog]: This master's current role is: FOLLOWER
W20260923 23:16:45.676883 6551 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f08b4fea623e461ebeb824f70032555c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 23:16:45.676977 6551 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 23:16:45.676682 6451 master_runner.cc:186] Error getting master registration for 127.24.227.254:39273: OK, instance_id { permanent_uuid: "7b15ad20e8b54b03aea340fd5795f929" instance_seqno: 1790205405368225 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260923 23:16:45.678685 6451 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Sep 23 23:16:45 dist-test-slave-754n krb5kdc[6288](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205405, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.227.253@KRBTEST.COM
I20260923 23:16:45.682034 6448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:45.674053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39204 (local address 127.24.227.253:38449)
0923 23:16:45.674084 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0923 23:16:45.674086 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:16:45.674094 (+ 8us) server_negotiation.cc:407] Connection header received
0923 23:16:45.674137 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:16:45.674138 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:16:45.674145 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:16:45.674257 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:16:45.674792 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:45.675487 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:45.676516 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:45.676791 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:45.676829 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:16:45.679256 (+ 2427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:16:45.679258 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:16:45.679258 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:16:45.679268 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:16:45.680896 (+ 1628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:16:45.681274 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:16:45.681275 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:16:45.681275 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:16:45.681343 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:16:45.681549 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:16:45.681550 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:16:45.681550 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:16:45.681668 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:16:45.681803 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0923 23:16:45.681882 (+ 79us) server_negotiation.cc:299] Negotiation successful
0923 23:16:45.681951 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260923 23:16:45.683660 25487 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 23:16:45.692167 6372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:45.685131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60220 (local address 127.24.227.254:39273)
0923 23:16:45.685164 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0923 23:16:45.685166 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:16:45.685401 (+ 235us) server_negotiation.cc:407] Connection header received
0923 23:16:45.685490 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:16:45.685491 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:16:45.685500 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:16:45.685528 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:16:45.686216 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:45.686859 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:45.687897 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:45.688176 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:45.688214 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:16:45.689545 (+ 1331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:16:45.689546 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:16:45.689547 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:16:45.689552 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:16:45.690966 (+ 1414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:16:45.691304 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:16:45.691305 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:16:45.691305 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:16:45.691379 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:16:45.691565 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:16:45.691565 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:16:45.691566 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:16:45.691672 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:16:45.691734 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0923 23:16:45.692000 (+ 266us) server_negotiation.cc:299] Negotiation successful
0923 23:16:45.692075 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260923 23:16:45.695626 6448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:45.685144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39218 (local address 127.24.227.253:38449)
0923 23:16:45.685177 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0923 23:16:45.685179 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:16:45.688024 (+ 2845us) server_negotiation.cc:407] Connection header received
0923 23:16:45.688113 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:16:45.688114 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:16:45.688123 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:16:45.688150 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:16:45.689080 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:45.689991 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:45.691120 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:45.691408 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:45.691446 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:16:45.692915 (+ 1469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:16:45.692917 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:16:45.692917 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:16:45.692923 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:16:45.694481 (+ 1558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:16:45.694833 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:16:45.694834 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:16:45.694835 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:16:45.694879 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:16:45.695086 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:16:45.695087 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:16:45.695087 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:16:45.695199 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:16:45.695264 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0923 23:16:45.695472 (+ 208us) server_negotiation.cc:299] Negotiation successful
0923 23:16:45.695540 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260923 23:16:45.699826 6523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:45.685483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59746 (local address 127.24.227.252:45233)
0923 23:16:45.685516 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0923 23:16:45.685518 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:16:45.692949 (+ 7431us) server_negotiation.cc:407] Connection header received
0923 23:16:45.693050 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:16:45.693051 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:16:45.693060 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:16:45.693098 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:16:45.693611 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:45.694241 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:45.695172 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:45.695488 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:45.695527 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:16:45.697163 (+ 1636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:16:45.697164 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:16:45.697165 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:16:45.697171 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:16:45.698733 (+ 1562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:16:45.699069 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:16:45.699070 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:16:45.699071 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:16:45.699115 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:16:45.699285 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:16:45.699286 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:16:45.699286 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:16:45.699402 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:16:45.699463 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0923 23:16:45.699671 (+ 208us) server_negotiation.cc:299] Negotiation successful
0923 23:16:45.699741 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260923 23:16:45.707427 6369 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.227.252" port: 45233 } attempt: 2
Sep 23 23:16:45 dist-test-slave-754n krb5kdc[6288](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205405, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.254@KRBTEST.COM for kudu/127.24.227.252@KRBTEST.COM
I20260923 23:16:45.715039 6371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:45.707669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.252:45233 (local address 127.0.0.1:59760)
0923 23:16:45.707711 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0923 23:16:45.707721 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 23:16:45.707754 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:16:45.707906 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:16:45.707907 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:16:45.708062 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:16:45.708102 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:16:45.708104 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:16:45.708931 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:16:45.708932 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:16:45.709662 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:16:45.709664 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:16:45.710111 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:16:45.710112 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:16:45.710168 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:16:45.710583 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:16:45.710588 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:16:45.712319 (+ 1731us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:16:45.713992 (+ 1673us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:16:45.713993 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:16:45.713993 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:16:45.714195 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:16:45.714390 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:16:45.714391 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:16:45.714391 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:16:45.714448 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:16:45.714711 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:16:45.714712 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:16:45.714763 (+ 51us) client_negotiation.cc:769] Sending connection context
0923 23:16:45.714848 (+ 85us) client_negotiation.cc:240] Negotiation successful
0923 23:16:45.714929 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260923 23:16:45.715059 6523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:45.707733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59760 (local address 127.24.227.252:45233)
0923 23:16:45.707766 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0923 23:16:45.707768 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:16:45.707775 (+ 7us) server_negotiation.cc:407] Connection header received
0923 23:16:45.707796 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:16:45.707797 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:16:45.707805 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:16:45.707904 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:16:45.708175 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:45.708852 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:45.709739 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:45.710035 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:45.710103 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:16:45.712408 (+ 2305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:16:45.712409 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:16:45.712409 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:16:45.712416 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:16:45.713912 (+ 1496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:16:45.714277 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:16:45.714278 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:16:45.714278 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:16:45.714329 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:16:45.714525 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:16:45.714525 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:16:45.714526 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:16:45.714643 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:16:45.714707 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0923 23:16:45.714881 (+ 174us) server_negotiation.cc:299] Negotiation successful
0923 23:16:45.714971 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260923 23:16:45.717018 6369 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b15ad20e8b54b03aea340fd5795f929: Bootstrap starting.
I20260923 23:16:45.717741 6369 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7b15ad20e8b54b03aea340fd5795f929: Neither blocks nor log segments found. Creating new log.
I20260923 23:16:45.718034 6369 log.cc:824] T 00000000000000000000000000000000 P 7b15ad20e8b54b03aea340fd5795f929: Log is configured to *not* fsync() on all Append() calls
I20260923 23:16:45.719152 6369 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b15ad20e8b54b03aea340fd5795f929: No bootstrap required, opened a new log
I20260923 23:16:45.721108 6369 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7b15ad20e8b54b03aea340fd5795f929 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b15ad20e8b54b03aea340fd5795f929" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 39273 } } peers { permanent_uuid: "3d13802e5bd44aa4a9979c188ca543cd" member_type: VOTER last_known_addr { host: "127.24.227.253" port: 38449 } } peers { permanent_uuid: "f08b4fea623e461ebeb824f70032555c" member_type: VOTER last_known_addr { host: "127.24.227.252" port: 45233 } }
I20260923 23:16:45.721232 6369 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7b15ad20e8b54b03aea340fd5795f929 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:16:45.721263 6369 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7b15ad20e8b54b03aea340fd5795f929 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b15ad20e8b54b03aea340fd5795f929, State: Initialized, Role: FOLLOWER
I20260923 23:16:45.721398 6369 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7b15ad20e8b54b03aea340fd5795f929 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b15ad20e8b54b03aea340fd5795f929" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 39273 } } peers { permanent_uuid: "3d13802e5bd44aa4a9979c188ca543cd" member_type: VOTER last_known_addr { host: "127.24.227.253" port: 38449 } } peers { permanent_uuid: "f08b4fea623e461ebeb824f70032555c" member_type: VOTER last_known_addr { host: "127.24.227.252" port: 45233 } }
I20260923 23:16:45.721688 6369 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7b15ad20e8b54b03aea340fd5795f929 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 23:16:45.721637 6561 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b15ad20e8b54b03aea340fd5795f929 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b15ad20e8b54b03aea340fd5795f929" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 39273 } } peers { permanent_uuid: "3d13802e5bd44aa4a9979c188ca543cd" member_type: VOTER last_known_addr { host: "127.24.227.253" port: 38449 } } peers { permanent_uuid: "f08b4fea623e461ebeb824f70032555c" member_type: VOTER last_known_addr { host: "127.24.227.252" port: 45233 } } }
I20260923 23:16:45.721817 6561 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b15ad20e8b54b03aea340fd5795f929 [sys.catalog]: This master's current role is: FOLLOWER
W20260923 23:16:45.723446 6572 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7b15ad20e8b54b03aea340fd5795f929: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 23:16:45.723539 6572 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 23:16:45.725471 6301 master_runner.cc:263] No leader master found from master 7b15ad20e8b54b03aea340fd5795f929
I20260923 23:16:45.725524 6301 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260923 23:16:45.731774 6540 raft_consensus.cc:482] T 00000000000000000000000000000000 P f08b4fea623e461ebeb824f70032555c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 23:16:45.731920 6540 raft_consensus.cc:504] T 00000000000000000000000000000000 P f08b4fea623e461ebeb824f70032555c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b15ad20e8b54b03aea340fd5795f929" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 39273 } } peers { permanent_uuid: "3d13802e5bd44aa4a9979c188ca543cd" member_type: VOTER last_known_addr { host: "127.24.227.253" port: 38449 } } peers { permanent_uuid: "f08b4fea623e461ebeb824f70032555c" member_type: VOTER last_known_addr { host: "127.24.227.252" port: 45233 } }
I20260923 23:16:45.732138 6540 leader_election.cc:288] T 00000000000000000000000000000000 P f08b4fea623e461ebeb824f70032555c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7b15ad20e8b54b03aea340fd5795f929 (127.24.227.254:39273), 3d13802e5bd44aa4a9979c188ca543cd (127.24.227.253:38449)
I20260923 23:16:45.732398 6418 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f08b4fea623e461ebeb824f70032555c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d13802e5bd44aa4a9979c188ca543cd" is_pre_election: true
I20260923 23:16:45.732524 6418 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3d13802e5bd44aa4a9979c188ca543cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f08b4fea623e461ebeb824f70032555c in term 0.
I20260923 23:16:45.732811 6343 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f08b4fea623e461ebeb824f70032555c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b15ad20e8b54b03aea340fd5795f929" is_pre_election: true
I20260923 23:16:45.732923 6343 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7b15ad20e8b54b03aea340fd5795f929 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f08b4fea623e461ebeb824f70032555c in term 0.
I20260923 23:16:45.733165 6470 leader_election.cc:302] T 00000000000000000000000000000000 P f08b4fea623e461ebeb824f70032555c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d13802e5bd44aa4a9979c188ca543cd, f08b4fea623e461ebeb824f70032555c; no voters:
I20260923 23:16:45.733341 6540 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f08b4fea623e461ebeb824f70032555c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 23:16:45.733392 6540 raft_consensus.cc:482] T 00000000000000000000000000000000 P f08b4fea623e461ebeb824f70032555c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 23:16:45.733418 6540 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f08b4fea623e461ebeb824f70032555c [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:16:45.734208 6540 raft_consensus.cc:504] T 00000000000000000000000000000000 P f08b4fea623e461ebeb824f70032555c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b15ad20e8b54b03aea340fd5795f929" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 39273 } } peers { permanent_uuid: "3d13802e5bd44aa4a9979c188ca543cd" member_type: VOTER last_known_addr { host: "127.24.227.253" port: 38449 } } peers { permanent_uuid: "f08b4fea623e461ebeb824f70032555c" member_type: VOTER last_known_addr { host: "127.24.227.252" port: 45233 } }
I20260923 23:16:45.734313 6540 leader_election.cc:288] T 00000000000000000000000000000000 P f08b4fea623e461ebeb824f70032555c [CANDIDATE]: Term 1 election: Requested vote from peers 7b15ad20e8b54b03aea340fd5795f929 (127.24.227.254:39273), 3d13802e5bd44aa4a9979c188ca543cd (127.24.227.253:38449)
I20260923 23:16:45.735236 6343 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f08b4fea623e461ebeb824f70032555c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b15ad20e8b54b03aea340fd5795f929"
I20260923 23:16:45.735311 6343 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7b15ad20e8b54b03aea340fd5795f929 [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:16:45.736167 6343 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7b15ad20e8b54b03aea340fd5795f929 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f08b4fea623e461ebeb824f70032555c in term 1.
I20260923 23:16:45.736317 6418 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f08b4fea623e461ebeb824f70032555c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d13802e5bd44aa4a9979c188ca543cd"
I20260923 23:16:45.736402 6418 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3d13802e5bd44aa4a9979c188ca543cd [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:16:45.737190 6418 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3d13802e5bd44aa4a9979c188ca543cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f08b4fea623e461ebeb824f70032555c in term 1.
I20260923 23:16:45.737390 6469 leader_election.cc:302] T 00000000000000000000000000000000 P f08b4fea623e461ebeb824f70032555c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7b15ad20e8b54b03aea340fd5795f929, f08b4fea623e461ebeb824f70032555c; no voters:
I20260923 23:16:45.737490 6540 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f08b4fea623e461ebeb824f70032555c [term 1 FOLLOWER]: Leader election won for term 1
I20260923 23:16:45.737622 6540 raft_consensus.cc:686] T 00000000000000000000000000000000 P f08b4fea623e461ebeb824f70032555c [term 1 LEADER]: Becoming Leader. State: Replica: f08b4fea623e461ebeb824f70032555c, State: Running, Role: LEADER
I20260923 23:16:45.737697 6540 consensus_queue.cc:237] T 00000000000000000000000000000000 P f08b4fea623e461ebeb824f70032555c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b15ad20e8b54b03aea340fd5795f929" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 39273 } } peers { permanent_uuid: "3d13802e5bd44aa4a9979c188ca543cd" member_type: VOTER last_known_addr { host: "127.24.227.253" port: 38449 } } peers { permanent_uuid: "f08b4fea623e461ebeb824f70032555c" member_type: VOTER last_known_addr { host: "127.24.227.252" port: 45233 } }
I20260923 23:16:45.737999 6540 sys_catalog.cc:455] T 00000000000000000000000000000000 P f08b4fea623e461ebeb824f70032555c [sys.catalog]: SysCatalogTable state changed. Reason: New leader f08b4fea623e461ebeb824f70032555c. Latest consensus state: current_term: 1 leader_uuid: "f08b4fea623e461ebeb824f70032555c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b15ad20e8b54b03aea340fd5795f929" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 39273 } } peers { permanent_uuid: "3d13802e5bd44aa4a9979c188ca543cd" member_type: VOTER last_known_addr { host: "127.24.227.253" port: 38449 } } peers { permanent_uuid: "f08b4fea623e461ebeb824f70032555c" member_type: VOTER last_known_addr { host: "127.24.227.252" port: 45233 } } }
I20260923 23:16:45.738063 6540 sys_catalog.cc:458] T 00000000000000000000000000000000 P f08b4fea623e461ebeb824f70032555c [sys.catalog]: This master's current role is: LEADER
I20260923 23:16:45.738236 6576 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 23:16:45.738402 6576 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 23:16:45.740729 6418 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3d13802e5bd44aa4a9979c188ca543cd [term 1 FOLLOWER]: Refusing update from remote peer f08b4fea623e461ebeb824f70032555c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 23:16:45.740976 6540 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f08b4fea623e461ebeb824f70032555c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d13802e5bd44aa4a9979c188ca543cd" member_type: VOTER last_known_addr { host: "127.24.227.253" port: 38449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 23:16:45.742122 6526 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d13802e5bd44aa4a9979c188ca543cd [sys.catalog]: SysCatalogTable state changed. Reason: New leader f08b4fea623e461ebeb824f70032555c. Latest consensus state: current_term: 1 leader_uuid: "f08b4fea623e461ebeb824f70032555c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b15ad20e8b54b03aea340fd5795f929" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 39273 } } peers { permanent_uuid: "3d13802e5bd44aa4a9979c188ca543cd" member_type: VOTER last_known_addr { host: "127.24.227.253" port: 38449 } } peers { permanent_uuid: "f08b4fea623e461ebeb824f70032555c" member_type: VOTER last_known_addr { host: "127.24.227.252" port: 45233 } } }
I20260923 23:16:45.742204 6526 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d13802e5bd44aa4a9979c188ca543cd [sys.catalog]: This master's current role is: FOLLOWER
I20260923 23:16:45.743153 6343 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7b15ad20e8b54b03aea340fd5795f929 [term 1 FOLLOWER]: Refusing update from remote peer f08b4fea623e461ebeb824f70032555c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 23:16:45.743377 6540 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f08b4fea623e461ebeb824f70032555c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7b15ad20e8b54b03aea340fd5795f929" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 39273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 23:16:45.744413 6561 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b15ad20e8b54b03aea340fd5795f929 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f08b4fea623e461ebeb824f70032555c. Latest consensus state: current_term: 1 leader_uuid: "f08b4fea623e461ebeb824f70032555c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b15ad20e8b54b03aea340fd5795f929" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 39273 } } peers { permanent_uuid: "3d13802e5bd44aa4a9979c188ca543cd" member_type: VOTER last_known_addr { host: "127.24.227.253" port: 38449 } } peers { permanent_uuid: "f08b4fea623e461ebeb824f70032555c" member_type: VOTER last_known_addr { host: "127.24.227.252" port: 45233 } } }
I20260923 23:16:45.744494 6561 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b15ad20e8b54b03aea340fd5795f929 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 23:16:45.748982 6540 sys_catalog.cc:455] T 00000000000000000000000000000000 P f08b4fea623e461ebeb824f70032555c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f08b4fea623e461ebeb824f70032555c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b15ad20e8b54b03aea340fd5795f929" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 39273 } } peers { permanent_uuid: "3d13802e5bd44aa4a9979c188ca543cd" member_type: VOTER last_known_addr { host: "127.24.227.253" port: 38449 } } peers { permanent_uuid: "f08b4fea623e461ebeb824f70032555c" member_type: VOTER last_known_addr { host: "127.24.227.252" port: 45233 } } }
I20260923 23:16:45.749064 6540 sys_catalog.cc:458] T 00000000000000000000000000000000 P f08b4fea623e461ebeb824f70032555c [sys.catalog]: This master's current role is: LEADER
I20260923 23:16:45.749455 6526 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d13802e5bd44aa4a9979c188ca543cd [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f08b4fea623e461ebeb824f70032555c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b15ad20e8b54b03aea340fd5795f929" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 39273 } } peers { permanent_uuid: "3d13802e5bd44aa4a9979c188ca543cd" member_type: VOTER last_known_addr { host: "127.24.227.253" port: 38449 } } peers { permanent_uuid: "f08b4fea623e461ebeb824f70032555c" member_type: VOTER last_known_addr { host: "127.24.227.252" port: 45233 } } }
I20260923 23:16:45.749537 6526 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d13802e5bd44aa4a9979c188ca543cd [sys.catalog]: This master's current role is: FOLLOWER
I20260923 23:16:45.749709 6577 mvcc.cc:205] Tried to move back new op lower bound from 7332681341911941120 to 7332681341902077952. Current Snapshot: MvccSnapshot[applied={T|T < 7332681341911941120}]
I20260923 23:16:45.751171 6574 sys_catalog.cc:455] T 00000000000000000000000000000000 P f08b4fea623e461ebeb824f70032555c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f08b4fea623e461ebeb824f70032555c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b15ad20e8b54b03aea340fd5795f929" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 39273 } } peers { permanent_uuid: "3d13802e5bd44aa4a9979c188ca543cd" member_type: VOTER last_known_addr { host: "127.24.227.253" port: 38449 } } peers { permanent_uuid: "f08b4fea623e461ebeb824f70032555c" member_type: VOTER last_known_addr { host: "127.24.227.252" port: 45233 } } }
I20260923 23:16:45.751273 6574 sys_catalog.cc:458] T 00000000000000000000000000000000 P f08b4fea623e461ebeb824f70032555c [sys.catalog]: This master's current role is: LEADER
I20260923 23:16:45.751699 6561 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b15ad20e8b54b03aea340fd5795f929 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f08b4fea623e461ebeb824f70032555c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b15ad20e8b54b03aea340fd5795f929" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 39273 } } peers { permanent_uuid: "3d13802e5bd44aa4a9979c188ca543cd" member_type: VOTER last_known_addr { host: "127.24.227.253" port: 38449 } } peers { permanent_uuid: "f08b4fea623e461ebeb824f70032555c" member_type: VOTER last_known_addr { host: "127.24.227.252" port: 45233 } } }
I20260923 23:16:45.751782 6561 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b15ad20e8b54b03aea340fd5795f929 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 23:16:45.752280 6576 catalog_manager.cc:1347] Generated new cluster ID: 21c3e3d94000409cacaa52299add538c
I20260923 23:16:45.752332 6576 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 23:16:45.752040 6580 mvcc.cc:205] Tried to move back new op lower bound from 7332681341911941120 to 7332681341902077952. Current Snapshot: MvccSnapshot[applied={T|T < 7332681341911941120}]
I20260923 23:16:45.791328 6576 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 23:16:45.792032 6576 catalog_manager.cc:1504] Loading token signing keys...
I20260923 23:16:45.798475 6576 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f08b4fea623e461ebeb824f70032555c: Generated new TSK 0
I20260923 23:16:45.798623 6576 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 23:16:46.666220 6537 catalog_manager.cc:1259] Loaded cluster ID: 21c3e3d94000409cacaa52299add538c
I20260923 23:16:46.666312 6537 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3d13802e5bd44aa4a9979c188ca543cd: loading cluster ID for follower catalog manager: success
I20260923 23:16:46.667054 6537 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3d13802e5bd44aa4a9979c188ca543cd: acquiring CA information for follower catalog manager: success
I20260923 23:16:46.667282 6537 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3d13802e5bd44aa4a9979c188ca543cd: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260923 23:16:46.724012 6572 catalog_manager.cc:1259] Loaded cluster ID: 21c3e3d94000409cacaa52299add538c
I20260923 23:16:46.724077 6572 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7b15ad20e8b54b03aea340fd5795f929: loading cluster ID for follower catalog manager: success
I20260923 23:16:46.724888 6572 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7b15ad20e8b54b03aea340fd5795f929: acquiring CA information for follower catalog manager: success
I20260923 23:16:46.725205 6572 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7b15ad20e8b54b03aea340fd5795f929: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260923 23:16:46.853797 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 6301
I20260923 23:16:46.859057 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 6376
I20260923 23:16:46.863473 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 6451
2026-09-23T23:16:46Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (2724 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790205259943518-25487-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 23:16:46 dist-test-slave-754n krb5kdc[6592](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 23:16:46 dist-test-slave-754n krb5kdc[6592](info): set up 2 sockets
Sep 23 23:16:46 dist-test-slave-754n krb5kdc[6592](info): commencing operation
krb5kdc: starting...
W20260923 23:16:47.867784 25487 mini_kdc.cc:121] Time spent starting KDC: real 0.985s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790205259943518-25487-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790205259943518-25487-0/krb5kdc/test-user.keytab.
Sep 23 23:16:47 dist-test-slave-754n krb5kdc[6592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205407, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T23:16:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T23:16:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.227.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.227.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.227.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab.
Entry for principal kudu/127.24.227.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab.
I20260923 23:16:47.976071 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790205259943518-25487-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790205259943518-25487-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790205259943518-25487-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.254:35437
--webserver_interface=127.24.227.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab
--principal=kudu/127.24.227.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:39807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.227.254:35437
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:16:48.053267 6605 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:16:48.053438 6605 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:16:48.053462 6605 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:16:48.054535 6605 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 23:16:48.054574 6605 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:16:48.054587 6605 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:16:48.054600 6605 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 23:16:48.054646 6605 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 23:16:48.056079 6605 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:39807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790205259943518-25487-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790205259943518-25487-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.227.254:35437
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab
--principal=kudu/127.24.227.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.254:35437
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.227.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6605
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790205259943518-25487-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:16:48.056448 6605 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:16:48.056699 6605 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:16:48.061415 6611 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:16:48.061442 6613 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:16:48.061517 6605 server_base.cc:1034] running on GCE node
W20260923 23:16:48.061424 6610 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:16:48.061808 6605 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:16:48.062083 6605 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:16:48.063247 6605 hybrid_clock.cc:648] HybridClock initialized: now 1790205408063230 us; error 27 us; skew 500 ppm
Sep 23 23:16:48 dist-test-slave-754n krb5kdc[6592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205408, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:16:48.065616 6605 init.cc:377] Logged in from keytab as kudu/127.24.227.254@KRBTEST.COM (short username kudu)
I20260923 23:16:48.066049 6605 webserver.cc:460] Webserver started at http://127.24.227.254:35333/ using document root <none> and password file <none>
I20260923 23:16:48.066269 6605 fs_manager.cc:359] Metadata directory not provided
I20260923 23:16:48.066332 6605 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:16:48.066476 6605 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:16:48.067446 6605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790205259943518-25487-0/minicluster-data/master-0/data/instance:
uuid: "abf780e1842042b3900cdd2c2a4940a7"
format_stamp: "Formatted at 2026-09-23 23:16:48 on dist-test-slave-754n"
server_key: "94305ecb67469f9898b24a16c2fa6188"
server_key_iv: "0957b77ea4e2df9858e8a959ee34b82e"
server_key_version: "encryptionkey@0"
I20260923 23:16:48.067770 6605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790205259943518-25487-0/minicluster-data/master-0/wal/instance:
uuid: "abf780e1842042b3900cdd2c2a4940a7"
format_stamp: "Formatted at 2026-09-23 23:16:48 on dist-test-slave-754n"
server_key: "94305ecb67469f9898b24a16c2fa6188"
server_key_iv: "0957b77ea4e2df9858e8a959ee34b82e"
server_key_version: "encryptionkey@0"
I20260923 23:16:48.069041 6605 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 23:16:48.069746 6620 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:16:48.069905 6605 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 23:16:48.070001 6605 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790205259943518-25487-0/minicluster-data/master-0/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790205259943518-25487-0/minicluster-data/master-0/wal
uuid: "abf780e1842042b3900cdd2c2a4940a7"
format_stamp: "Formatted at 2026-09-23 23:16:48 on dist-test-slave-754n"
server_key: "94305ecb67469f9898b24a16c2fa6188"
server_key_iv: "0957b77ea4e2df9858e8a959ee34b82e"
server_key_version: "encryptionkey@0"
I20260923 23:16:48.070073 6605 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790205259943518-25487-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790205259943518-25487-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790205259943518-25487-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:16:48.088811 6605 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:16:48.090517 6605 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:16:48.090677 6605 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:16:48.094504 6605 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.254:35437
I20260923 23:16:48.094535 6672 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.254:35437 every 8 connection(s)
I20260923 23:16:48.094861 6605 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
I20260923 23:16:48.095629 6673 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:16:48.097702 6673 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P abf780e1842042b3900cdd2c2a4940a7: Bootstrap starting.
I20260923 23:16:48.098187 6673 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P abf780e1842042b3900cdd2c2a4940a7: Neither blocks nor log segments found. Creating new log.
I20260923 23:16:48.098407 6673 log.cc:824] T 00000000000000000000000000000000 P abf780e1842042b3900cdd2c2a4940a7: Log is configured to *not* fsync() on all Append() calls
I20260923 23:16:48.099009 6673 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P abf780e1842042b3900cdd2c2a4940a7: No bootstrap required, opened a new log
I20260923 23:16:48.099803 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 6605
I20260923 23:16:48.099926 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790205259943518-25487-0/minicluster-data/master-0/wal/instance
I20260923 23:16:48.100114 25487 external_mini_cluster.cc:1428] Setting key be1a74e14d6cb5b2b298603ce8d04ba2
I20260923 23:16:48.101028 6673 raft_consensus.cc:348] T 00000000000000000000000000000000 P abf780e1842042b3900cdd2c2a4940a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abf780e1842042b3900cdd2c2a4940a7" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 35437 } }
I20260923 23:16:48.101188 6673 raft_consensus.cc:374] T 00000000000000000000000000000000 P abf780e1842042b3900cdd2c2a4940a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:16:48.101230 6673 raft_consensus.cc:729] T 00000000000000000000000000000000 P abf780e1842042b3900cdd2c2a4940a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abf780e1842042b3900cdd2c2a4940a7, State: Initialized, Role: FOLLOWER
I20260923 23:16:48.101361 6673 consensus_queue.cc:260] T 00000000000000000000000000000000 P abf780e1842042b3900cdd2c2a4940a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abf780e1842042b3900cdd2c2a4940a7" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 35437 } }
I20260923 23:16:48.101444 6673 raft_consensus.cc:388] T 00000000000000000000000000000000 P abf780e1842042b3900cdd2c2a4940a7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 23:16:48.101478 6673 raft_consensus.cc:482] T 00000000000000000000000000000000 P abf780e1842042b3900cdd2c2a4940a7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 23:16:48.101527 6673 raft_consensus.cc:3037] T 00000000000000000000000000000000 P abf780e1842042b3900cdd2c2a4940a7 [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:16:48.102416 6673 raft_consensus.cc:504] T 00000000000000000000000000000000 P abf780e1842042b3900cdd2c2a4940a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abf780e1842042b3900cdd2c2a4940a7" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 35437 } }
I20260923 23:16:48.102500 6673 leader_election.cc:302] T 00000000000000000000000000000000 P abf780e1842042b3900cdd2c2a4940a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: abf780e1842042b3900cdd2c2a4940a7; no voters:
I20260923 23:16:48.102658 6673 leader_election.cc:288] T 00000000000000000000000000000000 P abf780e1842042b3900cdd2c2a4940a7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 23:16:48.102699 6678 raft_consensus.cc:2781] T 00000000000000000000000000000000 P abf780e1842042b3900cdd2c2a4940a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 23:16:48.102847 6678 raft_consensus.cc:686] T 00000000000000000000000000000000 P abf780e1842042b3900cdd2c2a4940a7 [term 1 LEADER]: Becoming Leader. State: Replica: abf780e1842042b3900cdd2c2a4940a7, State: Running, Role: LEADER
I20260923 23:16:48.102903 6673 sys_catalog.cc:564] T 00000000000000000000000000000000 P abf780e1842042b3900cdd2c2a4940a7 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 23:16:48.102962 6678 consensus_queue.cc:237] T 00000000000000000000000000000000 P abf780e1842042b3900cdd2c2a4940a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abf780e1842042b3900cdd2c2a4940a7" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 35437 } }
I20260923 23:16:48.103376 6679 sys_catalog.cc:455] T 00000000000000000000000000000000 P abf780e1842042b3900cdd2c2a4940a7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "abf780e1842042b3900cdd2c2a4940a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abf780e1842042b3900cdd2c2a4940a7" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 35437 } } }
I20260923 23:16:48.103449 6679 sys_catalog.cc:458] T 00000000000000000000000000000000 P abf780e1842042b3900cdd2c2a4940a7 [sys.catalog]: This master's current role is: LEADER
I20260923 23:16:48.103650 6680 sys_catalog.cc:455] T 00000000000000000000000000000000 P abf780e1842042b3900cdd2c2a4940a7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader abf780e1842042b3900cdd2c2a4940a7. Latest consensus state: current_term: 1 leader_uuid: "abf780e1842042b3900cdd2c2a4940a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abf780e1842042b3900cdd2c2a4940a7" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 35437 } } }
I20260923 23:16:48.103739 6680 sys_catalog.cc:458] T 00000000000000000000000000000000 P abf780e1842042b3900cdd2c2a4940a7 [sys.catalog]: This master's current role is: LEADER
I20260923 23:16:48.104038 6686 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 23:16:48.104532 6686 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 23:16:48.105986 6686 catalog_manager.cc:1347] Generated new cluster ID: 5f67cc42cda243e6aca3d21def691eea
I20260923 23:16:48.106037 6686 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 23:16:48 dist-test-slave-754n krb5kdc[6592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205407, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
I20260923 23:16:48.110945 6677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:48.100543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49320 (local address 127.24.227.254:35437)
0923 23:16:48.100699 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0923 23:16:48.100712 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0923 23:16:48.100735 (+ 23us) server_negotiation.cc:407] Connection header received
0923 23:16:48.100991 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:16:48.100996 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:16:48.101035 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:16:48.101135 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:16:48.101884 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:48.102918 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:48.104017 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:48.104715 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:48.104819 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:16:48.107628 (+ 2809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:16:48.107631 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:16:48.107631 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:16:48.107642 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:16:48.109433 (+ 1791us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:16:48.109817 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:16:48.109820 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:16:48.109820 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:16:48.109928 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:16:48.110129 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:16:48.110129 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:16:48.110130 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:16:48.110383 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:16:48.110515 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0923 23:16:48.110649 (+ 134us) server_negotiation.cc:299] Negotiation successful
0923 23:16:48.110760 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":83,"threads_started":1}
I20260923 23:16:48.111953 25487 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 23:16:48.133132 6686 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 23:16:48.133798 6686 catalog_manager.cc:1504] Loading token signing keys...
I20260923 23:16:48.136263 6686 catalog_manager.cc:5868] T 00000000000000000000000000000000 P abf780e1842042b3900cdd2c2a4940a7: Generated new TSK 0
I20260923 23:16:48.136445 6686 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 23:16:48.165611 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 6605
2026-09-23T23:16:48Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1300 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 23:16:48 dist-test-slave-754n krb5kdc[6700](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 23:16:48 dist-test-slave-754n krb5kdc[6700](info): set up 2 sockets
Sep 23 23:16:48 dist-test-slave-754n krb5kdc[6700](info): commencing operation
krb5kdc: starting...
W20260923 23:16:49.179826 25487 mini_kdc.cc:121] Time spent starting KDC: real 0.997s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/krb5kdc/test-user.keytab.
Sep 23 23:16:49 dist-test-slave-754n krb5kdc[6700](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205409, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T23:16:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T23:16:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.227.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.227.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.227.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab.
Entry for principal kudu/127.24.227.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab.
I20260923 23:16:49.286691 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.254:41135
--webserver_interface=127.24.227.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab
--principal=kudu/127.24.227.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:33523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.227.254:41135
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:16:49.365564 6713 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:16:49.365744 6713 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:16:49.365765 6713 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:16:49.366878 6713 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 23:16:49.366916 6713 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:16:49.366933 6713 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:16:49.366978 6713 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 23:16:49.366997 6713 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 23:16:49.368484 6713 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:33523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.227.254:41135
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab
--principal=kudu/127.24.227.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.254:41135
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.227.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6713
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:16:49.368707 6713 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:16:49.368955 6713 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:16:49.373364 6719 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:16:49.373375 6721 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:16:49.373359 6718 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:16:49.373555 6713 server_base.cc:1034] running on GCE node
I20260923 23:16:49.373799 6713 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:16:49.374043 6713 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:16:49.375193 6713 hybrid_clock.cc:648] HybridClock initialized: now 1790205409375184 us; error 33 us; skew 500 ppm
Sep 23 23:16:49 dist-test-slave-754n krb5kdc[6700](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205409, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:16:49.386369 6713 init.cc:377] Logged in from keytab as kudu/127.24.227.254@KRBTEST.COM (short username kudu)
I20260923 23:16:49.386940 6713 webserver.cc:460] Webserver started at http://127.24.227.254:38573/ using document root <none> and password file <none>
I20260923 23:16:49.387223 6713 fs_manager.cc:359] Metadata directory not provided
I20260923 23:16:49.387283 6713 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:16:49.387421 6713 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:16:49.388857 6713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/master-0/data/instance:
uuid: "1c752c42c9f74c2491720c7a2b81880a"
format_stamp: "Formatted at 2026-09-23 23:16:49 on dist-test-slave-754n"
server_key: "b38b2c8c90194a9b4f4aa9b792948634"
server_key_iv: "e3b6e93484d1654b9ad3794cfba4b64f"
server_key_version: "encryptionkey@0"
I20260923 23:16:49.389297 6713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/master-0/wal/instance:
uuid: "1c752c42c9f74c2491720c7a2b81880a"
format_stamp: "Formatted at 2026-09-23 23:16:49 on dist-test-slave-754n"
server_key: "b38b2c8c90194a9b4f4aa9b792948634"
server_key_iv: "e3b6e93484d1654b9ad3794cfba4b64f"
server_key_version: "encryptionkey@0"
I20260923 23:16:49.391144 6713 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260923 23:16:49.392303 6728 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:16:49.392527 6713 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 23:16:49.392697 6713 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/master-0/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/master-0/wal
uuid: "1c752c42c9f74c2491720c7a2b81880a"
format_stamp: "Formatted at 2026-09-23 23:16:49 on dist-test-slave-754n"
server_key: "b38b2c8c90194a9b4f4aa9b792948634"
server_key_iv: "e3b6e93484d1654b9ad3794cfba4b64f"
server_key_version: "encryptionkey@0"
I20260923 23:16:49.392864 6713 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:16:49.407539 6713 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:16:49.410441 6713 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:16:49.410563 6713 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:16:49.414784 6713 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.254:41135
I20260923 23:16:49.414842 6780 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.254:41135 every 8 connection(s)
I20260923 23:16:49.415149 6713 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
I20260923 23:16:49.416479 6781 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:16:49.418582 6781 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c752c42c9f74c2491720c7a2b81880a: Bootstrap starting.
I20260923 23:16:49.419265 6781 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1c752c42c9f74c2491720c7a2b81880a: Neither blocks nor log segments found. Creating new log.
I20260923 23:16:49.419464 6781 log.cc:824] T 00000000000000000000000000000000 P 1c752c42c9f74c2491720c7a2b81880a: Log is configured to *not* fsync() on all Append() calls
I20260923 23:16:49.420224 6781 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c752c42c9f74c2491720c7a2b81880a: No bootstrap required, opened a new log
I20260923 23:16:49.420601 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 6713
I20260923 23:16:49.420691 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/master-0/wal/instance
I20260923 23:16:49.420898 25487 external_mini_cluster.cc:1428] Setting key 99a106a6ba3360b16560839db8beac1e
I20260923 23:16:49.422169 6781 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1c752c42c9f74c2491720c7a2b81880a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c752c42c9f74c2491720c7a2b81880a" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41135 } }
I20260923 23:16:49.422317 6781 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1c752c42c9f74c2491720c7a2b81880a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:16:49.422351 6781 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1c752c42c9f74c2491720c7a2b81880a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c752c42c9f74c2491720c7a2b81880a, State: Initialized, Role: FOLLOWER
I20260923 23:16:49.422549 6781 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1c752c42c9f74c2491720c7a2b81880a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c752c42c9f74c2491720c7a2b81880a" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41135 } }
I20260923 23:16:49.422639 6781 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1c752c42c9f74c2491720c7a2b81880a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 23:16:49.422672 6781 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1c752c42c9f74c2491720c7a2b81880a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 23:16:49.422727 6781 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1c752c42c9f74c2491720c7a2b81880a [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:16:49.423640 6781 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1c752c42c9f74c2491720c7a2b81880a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c752c42c9f74c2491720c7a2b81880a" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41135 } }
I20260923 23:16:49.423720 6781 leader_election.cc:302] T 00000000000000000000000000000000 P 1c752c42c9f74c2491720c7a2b81880a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1c752c42c9f74c2491720c7a2b81880a; no voters:
I20260923 23:16:49.423928 6786 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1c752c42c9f74c2491720c7a2b81880a [term 1 FOLLOWER]: Leader election won for term 1
I20260923 23:16:49.423955 6781 leader_election.cc:288] T 00000000000000000000000000000000 P 1c752c42c9f74c2491720c7a2b81880a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 23:16:49.424069 6786 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1c752c42c9f74c2491720c7a2b81880a [term 1 LEADER]: Becoming Leader. State: Replica: 1c752c42c9f74c2491720c7a2b81880a, State: Running, Role: LEADER
I20260923 23:16:49.424222 6781 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1c752c42c9f74c2491720c7a2b81880a [sys.catalog]: configured and running, proceeding with master startup.
I20260923 23:16:49.424202 6786 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1c752c42c9f74c2491720c7a2b81880a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c752c42c9f74c2491720c7a2b81880a" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41135 } }
I20260923 23:16:49.424634 6787 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c752c42c9f74c2491720c7a2b81880a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1c752c42c9f74c2491720c7a2b81880a. Latest consensus state: current_term: 1 leader_uuid: "1c752c42c9f74c2491720c7a2b81880a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c752c42c9f74c2491720c7a2b81880a" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41135 } } }
I20260923 23:16:49.424702 6787 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c752c42c9f74c2491720c7a2b81880a [sys.catalog]: This master's current role is: LEADER
I20260923 23:16:49.424861 6788 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c752c42c9f74c2491720c7a2b81880a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1c752c42c9f74c2491720c7a2b81880a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c752c42c9f74c2491720c7a2b81880a" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41135 } } }
I20260923 23:16:49.424973 6788 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c752c42c9f74c2491720c7a2b81880a [sys.catalog]: This master's current role is: LEADER
I20260923 23:16:49.425135 6793 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 23:16:49.425648 6793 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 23:16:49.427214 6793 catalog_manager.cc:1347] Generated new cluster ID: 49a5d989716d404a8f46fc88738ceaf5
I20260923 23:16:49.427268 6793 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 23:16:49 dist-test-slave-754n krb5kdc[6700](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205409, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
I20260923 23:16:49.437224 6785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:49.421310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41948 (local address 127.24.227.254:41135)
0923 23:16:49.421457 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0923 23:16:49.421464 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 23:16:49.421504 (+ 40us) server_negotiation.cc:407] Connection header received
0923 23:16:49.422146 (+ 642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:16:49.422151 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:16:49.422191 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:16:49.422291 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:16:49.423114 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:49.424200 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:49.425189 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:49.425914 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:49.426058 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:16:49.434098 (+ 8040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:16:49.434102 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:16:49.434102 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:16:49.434113 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:16:49.435670 (+ 1557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:16:49.436112 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:16:49.436115 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:16:49.436115 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:16:49.436200 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:16:49.436409 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:16:49.436410 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:16:49.436410 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:16:49.436630 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:16:49.436807 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0923 23:16:49.436873 (+ 66us) server_negotiation.cc:299] Negotiation successful
0923 23:16:49.437007 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":59,"threads_started":1}
I20260923 23:16:49.449950 6793 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 23:16:49.450402 6793 catalog_manager.cc:1504] Loading token signing keys...
I20260923 23:16:49.453364 6793 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1c752c42c9f74c2491720c7a2b81880a: Generated new TSK 0
I20260923 23:16:49.453529 6793 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.227.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.227.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.227.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.193.keytab.
Entry for principal kudu/127.24.227.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.193.keytab.
I20260923 23:16:49.488511 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.227.193:0
--local_ip_for_outbound_sockets=127.24.227.193
--webserver_interface=127.24.227.193
--webserver_port=0
--tserver_master_addrs=127.24.227.254:41135
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.193.keytab
--principal=kudu/127.24.227.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:33523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:16:49.577613 6807 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:16:49.577805 6807 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:16:49.577857 6807 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:16:49.578945 6807 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:16:49.579005 6807 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:16:49.579049 6807 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.227.193
I20260923 23:16:49.580533 6807 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:33523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.193.keytab
--principal=kudu/127.24.227.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.227.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.227.193
--webserver_port=0
--tserver_master_addrs=127.24.227.254:41135
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6807
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.227.193
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:16:49.580765 6807 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:16:49.580977 6807 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:16:49.585579 6815 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:16:49.585597 6812 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:16:49.585580 6813 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:16:49.585896 6807 server_base.cc:1034] running on GCE node
I20260923 23:16:49.586102 6807 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:16:49.586346 6807 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:16:49.587491 6807 hybrid_clock.cc:648] HybridClock initialized: now 1790205409587485 us; error 34 us; skew 500 ppm
Sep 23 23:16:49 dist-test-slave-754n krb5kdc[6700](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205409, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:16:49.589947 6807 init.cc:377] Logged in from keytab as kudu/127.24.227.193@KRBTEST.COM (short username kudu)
I20260923 23:16:49.590344 6807 webserver.cc:460] Webserver started at http://127.24.227.193:40821/ using document root <none> and password file <none>
I20260923 23:16:49.590550 6807 fs_manager.cc:359] Metadata directory not provided
I20260923 23:16:49.590593 6807 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:16:49.590744 6807 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:16:49.591686 6807 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-0/data/instance:
uuid: "06385beadc504000bfdbe0c62c0f09c2"
format_stamp: "Formatted at 2026-09-23 23:16:49 on dist-test-slave-754n"
server_key: "888e34ce2a70e59d063c8bbc4b1e42ae"
server_key_iv: "1be0c765f92a697e6dc06472fb31a01b"
server_key_version: "encryptionkey@0"
I20260923 23:16:49.592054 6807 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-0/wal/instance:
uuid: "06385beadc504000bfdbe0c62c0f09c2"
format_stamp: "Formatted at 2026-09-23 23:16:49 on dist-test-slave-754n"
server_key: "888e34ce2a70e59d063c8bbc4b1e42ae"
server_key_iv: "1be0c765f92a697e6dc06472fb31a01b"
server_key_version: "encryptionkey@0"
I20260923 23:16:49.593287 6807 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 23:16:49.593891 6822 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:16:49.594055 6807 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 23:16:49.594143 6807 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-0/wal
uuid: "06385beadc504000bfdbe0c62c0f09c2"
format_stamp: "Formatted at 2026-09-23 23:16:49 on dist-test-slave-754n"
server_key: "888e34ce2a70e59d063c8bbc4b1e42ae"
server_key_iv: "1be0c765f92a697e6dc06472fb31a01b"
server_key_version: "encryptionkey@0"
I20260923 23:16:49.594230 6807 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:16:49.607364 6807 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:16:49.609395 6807 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:16:49.609555 6807 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:16:49.609745 6807 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 23:16:49.610006 6807 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 23:16:49.610051 6807 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 23:16:49.610080 6807 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 23:16:49.610105 6807 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 23:16:49.615456 6807 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.193:44099
I20260923 23:16:49.615492 6935 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.193:44099 every 8 connection(s)
I20260923 23:16:49.615772 6807 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-0/data/info.pb
Sep 23 23:16:49 dist-test-slave-754n krb5kdc[6700](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205409, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.193@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
I20260923 23:16:49.623306 6785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:49.616416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.193:56203 (local address 127.24.227.254:41135)
0923 23:16:49.616468 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0923 23:16:49.616470 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:16:49.616658 (+ 188us) server_negotiation.cc:407] Connection header received
0923 23:16:49.617048 (+ 390us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:16:49.617049 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:16:49.617058 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:16:49.617103 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:16:49.617825 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:49.618301 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:49.619191 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:49.619364 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:49.619408 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:16:49.621174 (+ 1766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:16:49.621175 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:16:49.621175 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:16:49.621181 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:16:49.622270 (+ 1089us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:16:49.622613 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:16:49.622614 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:16:49.622614 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:16:49.622659 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:16:49.622843 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:16:49.622843 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:16:49.622844 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:16:49.622932 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:16:49.622976 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0923 23:16:49.623178 (+ 202us) server_negotiation.cc:299] Negotiation successful
0923 23:16:49.623234 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260923 23:16:49.623489 6938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:49.616406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41135 (local address 127.24.227.193:56203)
0923 23:16:49.616540 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0923 23:16:49.616583 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0923 23:16:49.616972 (+ 389us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:16:49.617144 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:16:49.617147 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:16:49.617608 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:16:49.617760 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:16:49.617764 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:16:49.618368 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:16:49.618368 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:16:49.619126 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:16:49.619127 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:16:49.619430 (+ 303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:16:49.619431 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:16:49.619515 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:16:49.619947 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:16:49.619954 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:16:49.621096 (+ 1142us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:16:49.622349 (+ 1253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:16:49.622355 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:16:49.622357 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:16:49.622556 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:16:49.622710 (+ 154us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:16:49.622710 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:16:49.622710 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:16:49.622796 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:16:49.622979 (+ 183us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:16:49.622983 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:16:49.623088 (+ 105us) client_negotiation.cc:769] Sending connection context
0923 23:16:49.623157 (+ 69us) client_negotiation.cc:240] Negotiation successful
0923 23:16:49.623251 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":74,"threads_started":1}
I20260923 23:16:49.623474 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 6807
I20260923 23:16:49.623571 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-0/wal/instance
I20260923 23:16:49.623765 25487 external_mini_cluster.cc:1428] Setting key a2a41ee4005acfb72c16a19661346884
I20260923 23:16:49.624064 6936 heartbeater.cc:347] Connected to a master server at 127.24.227.254:41135
I20260923 23:16:49.624162 6936 heartbeater.cc:464] Registering TS with master...
I20260923 23:16:49.624323 6936 heartbeater.cc:511] Master 127.24.227.254:41135 requested a full tablet report, sending...
I20260923 23:16:49.624883 6745 ts_manager.cc:194] Registered new tserver with Master: 06385beadc504000bfdbe0c62c0f09c2 (127.24.227.193:44099)
I20260923 23:16:49.625473 6745 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.227.193@KRBTEST.COM'} at 127.24.227.193:56203
WARNING: no policy specified for kudu/127.24.227.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.227.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.227.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.194.keytab.
Entry for principal kudu/127.24.227.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.194.keytab.
I20260923 23:16:49.646377 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.227.194:0
--local_ip_for_outbound_sockets=127.24.227.194
--webserver_interface=127.24.227.194
--webserver_port=0
--tserver_master_addrs=127.24.227.254:41135
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.194.keytab
--principal=kudu/127.24.227.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:33523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:16:49.722432 6941 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:16:49.722596 6941 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:16:49.722617 6941 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:16:49.723695 6941 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:16:49.723733 6941 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:16:49.723785 6941 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.227.194
I20260923 23:16:49.725311 6941 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:33523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.194.keytab
--principal=kudu/127.24.227.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.227.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.227.194
--webserver_port=0
--tserver_master_addrs=127.24.227.254:41135
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6941
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.227.194
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:16:49.725546 6941 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:16:49.725749 6941 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:16:49.730362 6949 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:16:49.730367 6947 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:16:49.730386 6941 server_base.cc:1034] running on GCE node
W20260923 23:16:49.730545 6946 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:16:49.730744 6941 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:16:49.730985 6941 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:16:49.732136 6941 hybrid_clock.cc:648] HybridClock initialized: now 1790205409732116 us; error 48 us; skew 500 ppm
Sep 23 23:16:49 dist-test-slave-754n krb5kdc[6700](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205409, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:16:49.734529 6941 init.cc:377] Logged in from keytab as kudu/127.24.227.194@KRBTEST.COM (short username kudu)
I20260923 23:16:49.734954 6941 webserver.cc:460] Webserver started at http://127.24.227.194:40821/ using document root <none> and password file <none>
I20260923 23:16:49.735172 6941 fs_manager.cc:359] Metadata directory not provided
I20260923 23:16:49.735215 6941 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:16:49.735369 6941 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:16:49.736397 6941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-1/data/instance:
uuid: "19e417d8de4a47d6b511eefbbe7043cf"
format_stamp: "Formatted at 2026-09-23 23:16:49 on dist-test-slave-754n"
server_key: "18f0ee9a2d01d9996329afa39a60c510"
server_key_iv: "f124b9ef428a1cb011648191998b03ab"
server_key_version: "encryptionkey@0"
I20260923 23:16:49.736748 6941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-1/wal/instance:
uuid: "19e417d8de4a47d6b511eefbbe7043cf"
format_stamp: "Formatted at 2026-09-23 23:16:49 on dist-test-slave-754n"
server_key: "18f0ee9a2d01d9996329afa39a60c510"
server_key_iv: "f124b9ef428a1cb011648191998b03ab"
server_key_version: "encryptionkey@0"
I20260923 23:16:49.737974 6941 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 23:16:49.738564 6956 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:16:49.738696 6941 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 23:16:49.738765 6941 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-1/wal
uuid: "19e417d8de4a47d6b511eefbbe7043cf"
format_stamp: "Formatted at 2026-09-23 23:16:49 on dist-test-slave-754n"
server_key: "18f0ee9a2d01d9996329afa39a60c510"
server_key_iv: "f124b9ef428a1cb011648191998b03ab"
server_key_version: "encryptionkey@0"
I20260923 23:16:49.738814 6941 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:16:49.750759 6941 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:16:49.752547 6941 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:16:49.752650 6941 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:16:49.752810 6941 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 23:16:49.753028 6941 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 23:16:49.753058 6941 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 23:16:49.753078 6941 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 23:16:49.753091 6941 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 23:16:49.758291 6941 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.194:34463
I20260923 23:16:49.758344 7069 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.194:34463 every 8 connection(s)
I20260923 23:16:49.758646 6941 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-1/data/info.pb
I20260923 23:16:49.760133 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 6941
I20260923 23:16:49.760234 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-1/wal/instance
I20260923 23:16:49.760414 25487 external_mini_cluster.cc:1428] Setting key 32dac4b0072bf3b349038589b04aef3a
Sep 23 23:16:49 dist-test-slave-754n krb5kdc[6700](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205409, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.194@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.24.227.195@KRBTEST.COM; defaulting to no policy
I20260923 23:16:49.768455 6785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:49.759348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.194:48205 (local address 127.24.227.254:41135)
0923 23:16:49.759431 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0923 23:16:49.759434 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 23:16:49.759583 (+ 149us) server_negotiation.cc:407] Connection header received
0923 23:16:49.760246 (+ 663us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:16:49.760248 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:16:49.760264 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:16:49.760324 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:16:49.761472 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:49.761945 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:49.763117 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:49.763416 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:49.763506 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:16:49.765784 (+ 2278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:16:49.765785 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:16:49.765785 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:16:49.765792 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:16:49.767107 (+ 1315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:16:49.767519 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:16:49.767520 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:16:49.767521 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:16:49.767576 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:16:49.767821 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:16:49.767822 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:16:49.767822 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:16:49.767977 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:16:49.768048 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0923 23:16:49.768293 (+ 245us) server_negotiation.cc:299] Negotiation successful
0923 23:16:49.768368 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260923 23:16:49.768640 7072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:49.759311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41135 (local address 127.24.227.194:48205)
0923 23:16:49.759477 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0923 23:16:49.759507 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0923 23:16:49.760141 (+ 634us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:16:49.760399 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:16:49.760403 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:16:49.761124 (+ 721us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:16:49.761386 (+ 262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:16:49.761393 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:16:49.762029 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:16:49.762030 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:16:49.763038 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:16:49.763040 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:16:49.763520 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:16:49.763521 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:16:49.763674 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:16:49.764147 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:16:49.764157 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:16:49.765674 (+ 1517us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:16:49.767205 (+ 1531us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:16:49.767212 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:16:49.767214 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:16:49.767433 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:16:49.767638 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:16:49.767639 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:16:49.767640 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:16:49.767758 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:16:49.768039 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:16:49.768045 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:16:49.768172 (+ 127us) client_negotiation.cc:769] Sending connection context
0923 23:16:49.768273 (+ 101us) client_negotiation.cc:240] Negotiation successful
0923 23:16:49.768414 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":76,"threads_started":1}
I20260923 23:16:49.769047 7070 heartbeater.cc:347] Connected to a master server at 127.24.227.254:41135
I20260923 23:16:49.769145 7070 heartbeater.cc:464] Registering TS with master...
I20260923 23:16:49.769321 7070 heartbeater.cc:511] Master 127.24.227.254:41135 requested a full tablet report, sending...
I20260923 23:16:49.769650 6745 ts_manager.cc:194] Registered new tserver with Master: 19e417d8de4a47d6b511eefbbe7043cf (127.24.227.194:34463)
I20260923 23:16:49.769968 6745 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.227.194@KRBTEST.COM'} at 127.24.227.194:48205
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.227.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.227.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.195.keytab.
Entry for principal kudu/127.24.227.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.195.keytab.
I20260923 23:16:49.782533 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.227.195:0
--local_ip_for_outbound_sockets=127.24.227.195
--webserver_interface=127.24.227.195
--webserver_port=0
--tserver_master_addrs=127.24.227.254:41135
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.195.keytab
--principal=kudu/127.24.227.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:33523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:16:49.860874 7075 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:16:49.861085 7075 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:16:49.861106 7075 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:16:49.862047 7075 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:16:49.862076 7075 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:16:49.862099 7075 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.227.195
I20260923 23:16:49.863461 7075 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:33523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.195.keytab
--principal=kudu/127.24.227.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.227.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.227.195
--webserver_port=0
--tserver_master_addrs=127.24.227.254:41135
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7075
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.227.195
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:16:49.863646 7075 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:16:49.863816 7075 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:16:49.868268 7083 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:16:49.868274 7080 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:16:49.868300 7081 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:16:49.868300 7075 server_base.cc:1034] running on GCE node
I20260923 23:16:49.868639 7075 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:16:49.868861 7075 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:16:49.870012 7075 hybrid_clock.cc:648] HybridClock initialized: now 1790205409869998 us; error 25 us; skew 500 ppm
Sep 23 23:16:49 dist-test-slave-754n krb5kdc[6700](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205409, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:16:49.872187 7075 init.cc:377] Logged in from keytab as kudu/127.24.227.195@KRBTEST.COM (short username kudu)
I20260923 23:16:49.872594 7075 webserver.cc:460] Webserver started at http://127.24.227.195:36787/ using document root <none> and password file <none>
I20260923 23:16:49.872802 7075 fs_manager.cc:359] Metadata directory not provided
I20260923 23:16:49.872861 7075 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:16:49.873023 7075 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:16:49.874013 7075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-2/data/instance:
uuid: "1dbd9c5173d840aca823fc791e513088"
format_stamp: "Formatted at 2026-09-23 23:16:49 on dist-test-slave-754n"
server_key: "9dfabea4b8a7cc86f59b5eeac6e2275e"
server_key_iv: "ed1e42305954f38553aae7d94e7497b9"
server_key_version: "encryptionkey@0"
I20260923 23:16:49.874346 7075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-2/wal/instance:
uuid: "1dbd9c5173d840aca823fc791e513088"
format_stamp: "Formatted at 2026-09-23 23:16:49 on dist-test-slave-754n"
server_key: "9dfabea4b8a7cc86f59b5eeac6e2275e"
server_key_iv: "ed1e42305954f38553aae7d94e7497b9"
server_key_version: "encryptionkey@0"
I20260923 23:16:49.875566 7075 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 23:16:49.876298 7090 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:16:49.876448 7075 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 23:16:49.876536 7075 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-2/wal
uuid: "1dbd9c5173d840aca823fc791e513088"
format_stamp: "Formatted at 2026-09-23 23:16:49 on dist-test-slave-754n"
server_key: "9dfabea4b8a7cc86f59b5eeac6e2275e"
server_key_iv: "ed1e42305954f38553aae7d94e7497b9"
server_key_version: "encryptionkey@0"
I20260923 23:16:49.876606 7075 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:16:49.883414 7075 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:16:49.885048 7075 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:16:49.885186 7075 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:16:49.885375 7075 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 23:16:49.885649 7075 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 23:16:49.885697 7075 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 23:16:49.885735 7075 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 23:16:49.885763 7075 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 23:16:49.891292 7075 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.195:36069
I20260923 23:16:49.891332 7203 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.195:36069 every 8 connection(s)
I20260923 23:16:49.891592 7075 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-2/data/info.pb
I20260923 23:16:49.896522 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 7075
I20260923 23:16:49.896623 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-2/wal/instance
I20260923 23:16:49.896795 25487 external_mini_cluster.cc:1428] Setting key b7d0948e928de6acdfb174c0ecc80d74
Sep 23 23:16:49 dist-test-slave-754n krb5kdc[6700](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205409, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.195@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
I20260923 23:16:49.900381 6785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:49.892255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.195:43425 (local address 127.24.227.254:41135)
0923 23:16:49.892295 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0923 23:16:49.892297 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:16:49.892480 (+ 183us) server_negotiation.cc:407] Connection header received
0923 23:16:49.892944 (+ 464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:16:49.892945 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:16:49.892959 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:16:49.893014 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:16:49.893901 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:49.894422 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:49.895410 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:49.895617 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:49.895699 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:16:49.898110 (+ 2411us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:16:49.898111 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:16:49.898111 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:16:49.898118 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:16:49.899211 (+ 1093us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:16:49.899556 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:16:49.899556 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:16:49.899557 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:16:49.899602 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:16:49.899764 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:16:49.899764 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:16:49.899765 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:16:49.899856 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:16:49.899964 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0923 23:16:49.900256 (+ 292us) server_negotiation.cc:299] Negotiation successful
0923 23:16:49.900321 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260923 23:16:49.900508 7206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:49.892236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41135 (local address 127.24.227.195:43425)
0923 23:16:49.892370 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0923 23:16:49.892401 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0923 23:16:49.892840 (+ 439us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:16:49.893073 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:16:49.893077 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:16:49.893640 (+ 563us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:16:49.893801 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:16:49.893804 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:16:49.894531 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:16:49.894533 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:16:49.895315 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:16:49.895317 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:16:49.895709 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:16:49.895709 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:16:49.895832 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:16:49.896274 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:16:49.896281 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:16:49.898029 (+ 1748us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:16:49.899273 (+ 1244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:16:49.899279 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:16:49.899280 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:16:49.899482 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:16:49.899640 (+ 158us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:16:49.899640 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:16:49.899641 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:16:49.899714 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:16:49.899995 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:16:49.900000 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:16:49.900139 (+ 139us) client_negotiation.cc:769] Sending connection context
0923 23:16:49.900219 (+ 80us) client_negotiation.cc:240] Negotiation successful
0923 23:16:49.900327 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":70,"threads_started":1}
I20260923 23:16:49.900924 7204 heartbeater.cc:347] Connected to a master server at 127.24.227.254:41135
I20260923 23:16:49.901024 7204 heartbeater.cc:464] Registering TS with master...
I20260923 23:16:49.901180 7204 heartbeater.cc:511] Master 127.24.227.254:41135 requested a full tablet report, sending...
I20260923 23:16:49.901515 6745 ts_manager.cc:194] Registered new tserver with Master: 1dbd9c5173d840aca823fc791e513088 (127.24.227.195:36069)
I20260923 23:16:49.901853 6745 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.227.195@KRBTEST.COM'} at 127.24.227.195:43425
I20260923 23:16:49.908514 25487 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 23:16:49.914951 6785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:49.909703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41960 (local address 127.24.227.254:41135)
0923 23:16:49.909749 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0923 23:16:49.909751 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:16:49.909843 (+ 92us) server_negotiation.cc:407] Connection header received
0923 23:16:49.909896 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:16:49.909896 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:16:49.909904 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:16:49.909961 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:16:49.910476 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:49.910948 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:49.911619 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:49.911788 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:49.911833 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:16:49.912930 (+ 1097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:16:49.912931 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:16:49.912931 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:16:49.912938 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:16:49.914066 (+ 1128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:16:49.914341 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:16:49.914341 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:16:49.914342 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:16:49.914386 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:16:49.914527 (+ 141us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:16:49.914528 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:16:49.914528 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:16:49.914657 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:16:49.914723 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0923 23:16:49.914805 (+ 82us) server_negotiation.cc:299] Negotiation successful
0923 23:16:49.914858 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 23:16:49.916553 6745 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41960:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 23:16:49.917996 6745 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 23:16:49.924005 7217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:49.920465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54562 (local address 127.24.227.193:44099)
0923 23:16:49.920691 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0923 23:16:49.920696 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 23:16:49.920704 (+ 8us) server_negotiation.cc:407] Connection header received
0923 23:16:49.920740 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:16:49.920742 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:16:49.920758 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:16:49.920966 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:16:49.921033 (+ 67us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:49.921936 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:49.923009 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:49.923645 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:49.923723 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:16:49.923771 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0923 23:16:49.923844 (+ 73us) server_negotiation.cc:299] Negotiation successful
0923 23:16:49.923926 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":94,"threads_started":1}
I20260923 23:16:49.924002 7214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:49.920135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.193:44099 (local address 127.0.0.1:54562)
0923 23:16:49.920219 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0923 23:16:49.920229 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 23:16:49.920281 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:16:49.920855 (+ 574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:16:49.920856 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:16:49.920858 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:16:49.920903 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:16:49.920905 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:16:49.922031 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:16:49.922031 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:16:49.922898 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:16:49.922902 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:16:49.923721 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:16:49.923722 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:16:49.923770 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:16:49.923773 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 23:16:49.923830 (+ 57us) client_negotiation.cc:240] Negotiation successful
0923 23:16:49.923861 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71,"thread_start_us":51,"threads_started":1}
I20260923 23:16:49.924063 7216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:49.920343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43998 (local address 127.24.227.195:36069)
0923 23:16:49.920528 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0923 23:16:49.920532 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 23:16:49.920558 (+ 26us) server_negotiation.cc:407] Connection header received
0923 23:16:49.920590 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:16:49.920591 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:16:49.920604 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:16:49.920689 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:16:49.920875 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:49.921530 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:49.923018 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:49.923471 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:49.923552 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:16:49.923606 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0923 23:16:49.923936 (+ 330us) server_negotiation.cc:299] Negotiation successful
0923 23:16:49.923973 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":65,"threads_started":1}
I20260923 23:16:49.923851 7215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:49.920266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.195:36069 (local address 127.0.0.1:43998)
0923 23:16:49.920429 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0923 23:16:49.920438 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 23:16:49.920497 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:16:49.920759 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:16:49.920760 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:16:49.920762 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:16:49.920802 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:16:49.920803 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:16:49.921627 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:16:49.921628 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:16:49.922959 (+ 1331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:16:49.922961 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:16:49.923563 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:16:49.923564 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:16:49.923640 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:16:49.923652 (+ 12us) client_negotiation.cc:769] Sending connection context
0923 23:16:49.923714 (+ 62us) client_negotiation.cc:240] Negotiation successful
0923 23:16:49.923759 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"mutex_wait_us":28,"thread_start_us":38,"threads_started":1}
I20260923 23:16:49.924407 7213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:49.920002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37976 (local address 127.24.227.194:34463)
0923 23:16:49.920156 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0923 23:16:49.920161 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 23:16:49.920170 (+ 9us) server_negotiation.cc:407] Connection header received
0923 23:16:49.920209 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:16:49.920211 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:16:49.920227 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:16:49.920306 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:16:49.920571 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:49.921669 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:49.922965 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:49.923645 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:49.923723 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:16:49.923771 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0923 23:16:49.924204 (+ 433us) server_negotiation.cc:299] Negotiation successful
0923 23:16:49.924271 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":94,"threads_started":1}
I20260923 23:16:49.924598 7212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:49.919902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.194:34463 (local address 127.0.0.1:37976)
0923 23:16:49.920028 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0923 23:16:49.920066 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0923 23:16:49.920124 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:16:49.920392 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:16:49.920395 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:16:49.920397 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:16:49.920467 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:16:49.920469 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:16:49.921763 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:16:49.921764 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:16:49.922883 (+ 1119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:16:49.922886 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:16:49.923936 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:16:49.923937 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:16:49.923987 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:16:49.923990 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 23:16:49.924040 (+ 50us) client_negotiation.cc:240] Negotiation successful
0923 23:16:49.924075 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"mutex_wait_us":10,"thread_start_us":89,"threads_started":1}
I20260923 23:16:49.925038 7138 tablet_service.cc:1467] Processing CreateTablet for tablet 79757d715dae41c8ab81c7ff45013b89 (DEFAULT_TABLE table=test-table [id=1c20268fa0404d95a6444446f5156370]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 23:16:49.925413 7138 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79757d715dae41c8ab81c7ff45013b89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:16:49.925361 7004 tablet_service.cc:1467] Processing CreateTablet for tablet 79757d715dae41c8ab81c7ff45013b89 (DEFAULT_TABLE table=test-table [id=1c20268fa0404d95a6444446f5156370]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 23:16:49.925681 7004 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79757d715dae41c8ab81c7ff45013b89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:16:49.927947 7218 tablet_bootstrap.cc:492] T 79757d715dae41c8ab81c7ff45013b89 P 1dbd9c5173d840aca823fc791e513088: Bootstrap starting.
I20260923 23:16:49.928371 7219 tablet_bootstrap.cc:492] T 79757d715dae41c8ab81c7ff45013b89 P 19e417d8de4a47d6b511eefbbe7043cf: Bootstrap starting.
I20260923 23:16:49.928581 7218 tablet_bootstrap.cc:654] T 79757d715dae41c8ab81c7ff45013b89 P 1dbd9c5173d840aca823fc791e513088: Neither blocks nor log segments found. Creating new log.
I20260923 23:16:49.928699 6870 tablet_service.cc:1467] Processing CreateTablet for tablet 79757d715dae41c8ab81c7ff45013b89 (DEFAULT_TABLE table=test-table [id=1c20268fa0404d95a6444446f5156370]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 23:16:49.928967 7219 tablet_bootstrap.cc:654] T 79757d715dae41c8ab81c7ff45013b89 P 19e417d8de4a47d6b511eefbbe7043cf: Neither blocks nor log segments found. Creating new log.
I20260923 23:16:49.928967 6870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79757d715dae41c8ab81c7ff45013b89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:16:49.928998 7218 log.cc:824] T 79757d715dae41c8ab81c7ff45013b89 P 1dbd9c5173d840aca823fc791e513088: Log is configured to *not* fsync() on all Append() calls
I20260923 23:16:49.929477 7219 log.cc:824] T 79757d715dae41c8ab81c7ff45013b89 P 19e417d8de4a47d6b511eefbbe7043cf: Log is configured to *not* fsync() on all Append() calls
I20260923 23:16:49.929587 7218 tablet_bootstrap.cc:492] T 79757d715dae41c8ab81c7ff45013b89 P 1dbd9c5173d840aca823fc791e513088: No bootstrap required, opened a new log
I20260923 23:16:49.929627 7218 ts_tablet_manager.cc:1397] T 79757d715dae41c8ab81c7ff45013b89 P 1dbd9c5173d840aca823fc791e513088: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 23:16:49.930217 7219 tablet_bootstrap.cc:492] T 79757d715dae41c8ab81c7ff45013b89 P 19e417d8de4a47d6b511eefbbe7043cf: No bootstrap required, opened a new log
I20260923 23:16:49.930286 7219 ts_tablet_manager.cc:1397] T 79757d715dae41c8ab81c7ff45013b89 P 19e417d8de4a47d6b511eefbbe7043cf: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 23:16:49.930938 7218 raft_consensus.cc:348] T 79757d715dae41c8ab81c7ff45013b89 P 1dbd9c5173d840aca823fc791e513088 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19e417d8de4a47d6b511eefbbe7043cf" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 34463 } } peers { permanent_uuid: "06385beadc504000bfdbe0c62c0f09c2" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 44099 } } peers { permanent_uuid: "1dbd9c5173d840aca823fc791e513088" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 36069 } }
I20260923 23:16:49.931043 7218 raft_consensus.cc:374] T 79757d715dae41c8ab81c7ff45013b89 P 1dbd9c5173d840aca823fc791e513088 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:16:49.931061 7218 raft_consensus.cc:729] T 79757d715dae41c8ab81c7ff45013b89 P 1dbd9c5173d840aca823fc791e513088 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dbd9c5173d840aca823fc791e513088, State: Initialized, Role: FOLLOWER
I20260923 23:16:49.931205 7218 consensus_queue.cc:260] T 79757d715dae41c8ab81c7ff45013b89 P 1dbd9c5173d840aca823fc791e513088 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19e417d8de4a47d6b511eefbbe7043cf" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 34463 } } peers { permanent_uuid: "06385beadc504000bfdbe0c62c0f09c2" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 44099 } } peers { permanent_uuid: "1dbd9c5173d840aca823fc791e513088" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 36069 } }
I20260923 23:16:49.931433 7204 heartbeater.cc:503] Master 127.24.227.254:41135 was elected leader, sending a full tablet report...
I20260923 23:16:49.931444 7218 ts_tablet_manager.cc:1428] T 79757d715dae41c8ab81c7ff45013b89 P 1dbd9c5173d840aca823fc791e513088: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260923 23:16:49.932179 7219 raft_consensus.cc:348] T 79757d715dae41c8ab81c7ff45013b89 P 19e417d8de4a47d6b511eefbbe7043cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19e417d8de4a47d6b511eefbbe7043cf" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 34463 } } peers { permanent_uuid: "06385beadc504000bfdbe0c62c0f09c2" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 44099 } } peers { permanent_uuid: "1dbd9c5173d840aca823fc791e513088" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 36069 } }
I20260923 23:16:49.932335 7219 raft_consensus.cc:374] T 79757d715dae41c8ab81c7ff45013b89 P 19e417d8de4a47d6b511eefbbe7043cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:16:49.932377 7219 raft_consensus.cc:729] T 79757d715dae41c8ab81c7ff45013b89 P 19e417d8de4a47d6b511eefbbe7043cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19e417d8de4a47d6b511eefbbe7043cf, State: Initialized, Role: FOLLOWER
I20260923 23:16:49.932487 7219 consensus_queue.cc:260] T 79757d715dae41c8ab81c7ff45013b89 P 19e417d8de4a47d6b511eefbbe7043cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19e417d8de4a47d6b511eefbbe7043cf" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 34463 } } peers { permanent_uuid: "06385beadc504000bfdbe0c62c0f09c2" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 44099 } } peers { permanent_uuid: "1dbd9c5173d840aca823fc791e513088" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 36069 } }
I20260923 23:16:49.932731 7219 ts_tablet_manager.cc:1428] T 79757d715dae41c8ab81c7ff45013b89 P 19e417d8de4a47d6b511eefbbe7043cf: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260923 23:16:49.932773 7070 heartbeater.cc:503] Master 127.24.227.254:41135 was elected leader, sending a full tablet report...
I20260923 23:16:49.932878 7223 tablet_bootstrap.cc:492] T 79757d715dae41c8ab81c7ff45013b89 P 06385beadc504000bfdbe0c62c0f09c2: Bootstrap starting.
I20260923 23:16:49.933514 7223 tablet_bootstrap.cc:654] T 79757d715dae41c8ab81c7ff45013b89 P 06385beadc504000bfdbe0c62c0f09c2: Neither blocks nor log segments found. Creating new log.
I20260923 23:16:49.934023 7223 log.cc:824] T 79757d715dae41c8ab81c7ff45013b89 P 06385beadc504000bfdbe0c62c0f09c2: Log is configured to *not* fsync() on all Append() calls
I20260923 23:16:49.934675 7223 tablet_bootstrap.cc:492] T 79757d715dae41c8ab81c7ff45013b89 P 06385beadc504000bfdbe0c62c0f09c2: No bootstrap required, opened a new log
I20260923 23:16:49.934738 7223 ts_tablet_manager.cc:1397] T 79757d715dae41c8ab81c7ff45013b89 P 06385beadc504000bfdbe0c62c0f09c2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 23:16:49.936133 7223 raft_consensus.cc:348] T 79757d715dae41c8ab81c7ff45013b89 P 06385beadc504000bfdbe0c62c0f09c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19e417d8de4a47d6b511eefbbe7043cf" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 34463 } } peers { permanent_uuid: "06385beadc504000bfdbe0c62c0f09c2" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 44099 } } peers { permanent_uuid: "1dbd9c5173d840aca823fc791e513088" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 36069 } }
I20260923 23:16:49.936246 7223 raft_consensus.cc:374] T 79757d715dae41c8ab81c7ff45013b89 P 06385beadc504000bfdbe0c62c0f09c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:16:49.936285 7223 raft_consensus.cc:729] T 79757d715dae41c8ab81c7ff45013b89 P 06385beadc504000bfdbe0c62c0f09c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06385beadc504000bfdbe0c62c0f09c2, State: Initialized, Role: FOLLOWER
I20260923 23:16:49.936389 7223 consensus_queue.cc:260] T 79757d715dae41c8ab81c7ff45013b89 P 06385beadc504000bfdbe0c62c0f09c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19e417d8de4a47d6b511eefbbe7043cf" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 34463 } } peers { permanent_uuid: "06385beadc504000bfdbe0c62c0f09c2" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 44099 } } peers { permanent_uuid: "1dbd9c5173d840aca823fc791e513088" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 36069 } }
I20260923 23:16:49.936673 7223 ts_tablet_manager.cc:1428] T 79757d715dae41c8ab81c7ff45013b89 P 06385beadc504000bfdbe0c62c0f09c2: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260923 23:16:49.936687 6936 heartbeater.cc:503] Master 127.24.227.254:41135 was elected leader, sending a full tablet report...
W20260923 23:16:50.009176 7071 tablet.cc:2369] T 79757d715dae41c8ab81c7ff45013b89 P 19e417d8de4a47d6b511eefbbe7043cf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 23:16:50.065281 7224 raft_consensus.cc:482] T 79757d715dae41c8ab81c7ff45013b89 P 19e417d8de4a47d6b511eefbbe7043cf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 23:16:50.065438 7224 raft_consensus.cc:504] T 79757d715dae41c8ab81c7ff45013b89 P 19e417d8de4a47d6b511eefbbe7043cf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19e417d8de4a47d6b511eefbbe7043cf" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 34463 } } peers { permanent_uuid: "06385beadc504000bfdbe0c62c0f09c2" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 44099 } } peers { permanent_uuid: "1dbd9c5173d840aca823fc791e513088" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 36069 } }
I20260923 23:16:50.065821 7224 leader_election.cc:288] T 79757d715dae41c8ab81c7ff45013b89 P 19e417d8de4a47d6b511eefbbe7043cf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06385beadc504000bfdbe0c62c0f09c2 (127.24.227.193:44099), 1dbd9c5173d840aca823fc791e513088 (127.24.227.195:36069)
I20260923 23:16:50.069797 7217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:50.066115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.194:46723 (local address 127.24.227.193:44099)
0923 23:16:50.066168 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0923 23:16:50.066170 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:16:50.066180 (+ 10us) server_negotiation.cc:407] Connection header received
0923 23:16:50.066209 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:16:50.066210 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:16:50.066222 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:16:50.066279 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:16:50.066475 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:50.067232 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:50.068822 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:50.069445 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:50.069517 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:16:50.069523 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0923 23:16:50.069691 (+ 168us) server_negotiation.cc:299] Negotiation successful
0923 23:16:50.069728 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260923 23:16:50.069797 7216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:50.066249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.194:34111 (local address 127.24.227.195:36069)
0923 23:16:50.066302 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0923 23:16:50.066304 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:16:50.066342 (+ 38us) server_negotiation.cc:407] Connection header received
0923 23:16:50.066406 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:16:50.066407 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:16:50.066414 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:16:50.066483 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:16:50.066603 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:50.067283 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:50.068822 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:50.069434 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:50.069514 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:16:50.069523 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0923 23:16:50.069691 (+ 168us) server_negotiation.cc:299] Negotiation successful
0923 23:16:50.069728 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260923 23:16:50.069820 7072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:50.065990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.193:44099 (local address 127.24.227.194:46723)
0923 23:16:50.066048 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0923 23:16:50.066061 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 23:16:50.066127 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:16:50.066321 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:16:50.066322 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:16:50.066324 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:16:50.066399 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:16:50.066401 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:16:50.067311 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:16:50.067312 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:16:50.068711 (+ 1399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:16:50.068714 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:16:50.069524 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:16:50.069525 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:16:50.069606 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:16:50.069607 (+ 1us) client_negotiation.cc:769] Sending connection context
0923 23:16:50.069671 (+ 64us) client_negotiation.cc:240] Negotiation successful
0923 23:16:50.069721 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35,"mutex_wait_us":73}
I20260923 23:16:50.069826 7227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:50.066108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.195:36069 (local address 127.24.227.194:34111)
0923 23:16:50.066277 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0923 23:16:50.066288 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 23:16:50.066342 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:16:50.066483 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:16:50.066484 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:16:50.066485 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:16:50.066540 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:16:50.066541 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:16:50.067358 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:16:50.067359 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:16:50.068711 (+ 1352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:16:50.068714 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:16:50.069525 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:16:50.069526 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:16:50.069602 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:16:50.069606 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 23:16:50.069671 (+ 65us) client_negotiation.cc:240] Negotiation successful
0923 23:16:50.069721 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"mutex_wait_us":38,"thread_start_us":107,"threads_started":1}
I20260923 23:16:50.070211 7158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79757d715dae41c8ab81c7ff45013b89" candidate_uuid: "19e417d8de4a47d6b511eefbbe7043cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dbd9c5173d840aca823fc791e513088" is_pre_election: true
I20260923 23:16:50.070294 6890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79757d715dae41c8ab81c7ff45013b89" candidate_uuid: "19e417d8de4a47d6b511eefbbe7043cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06385beadc504000bfdbe0c62c0f09c2" is_pre_election: true
I20260923 23:16:50.070374 7158 raft_consensus.cc:2454] T 79757d715dae41c8ab81c7ff45013b89 P 1dbd9c5173d840aca823fc791e513088 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19e417d8de4a47d6b511eefbbe7043cf in term 0.
I20260923 23:16:50.070483 6890 raft_consensus.cc:2454] T 79757d715dae41c8ab81c7ff45013b89 P 06385beadc504000bfdbe0c62c0f09c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19e417d8de4a47d6b511eefbbe7043cf in term 0.
I20260923 23:16:50.070614 6960 leader_election.cc:302] T 79757d715dae41c8ab81c7ff45013b89 P 19e417d8de4a47d6b511eefbbe7043cf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19e417d8de4a47d6b511eefbbe7043cf, 1dbd9c5173d840aca823fc791e513088; no voters:
I20260923 23:16:50.070735 7224 raft_consensus.cc:2781] T 79757d715dae41c8ab81c7ff45013b89 P 19e417d8de4a47d6b511eefbbe7043cf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 23:16:50.070799 7224 raft_consensus.cc:482] T 79757d715dae41c8ab81c7ff45013b89 P 19e417d8de4a47d6b511eefbbe7043cf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 23:16:50.070829 7224 raft_consensus.cc:3037] T 79757d715dae41c8ab81c7ff45013b89 P 19e417d8de4a47d6b511eefbbe7043cf [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:16:50.071496 7224 raft_consensus.cc:504] T 79757d715dae41c8ab81c7ff45013b89 P 19e417d8de4a47d6b511eefbbe7043cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19e417d8de4a47d6b511eefbbe7043cf" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 34463 } } peers { permanent_uuid: "06385beadc504000bfdbe0c62c0f09c2" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 44099 } } peers { permanent_uuid: "1dbd9c5173d840aca823fc791e513088" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 36069 } }
I20260923 23:16:50.071617 7224 leader_election.cc:288] T 79757d715dae41c8ab81c7ff45013b89 P 19e417d8de4a47d6b511eefbbe7043cf [CANDIDATE]: Term 1 election: Requested vote from peers 06385beadc504000bfdbe0c62c0f09c2 (127.24.227.193:44099), 1dbd9c5173d840aca823fc791e513088 (127.24.227.195:36069)
I20260923 23:16:50.071772 6890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79757d715dae41c8ab81c7ff45013b89" candidate_uuid: "19e417d8de4a47d6b511eefbbe7043cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06385beadc504000bfdbe0c62c0f09c2"
I20260923 23:16:50.071794 7158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79757d715dae41c8ab81c7ff45013b89" candidate_uuid: "19e417d8de4a47d6b511eefbbe7043cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dbd9c5173d840aca823fc791e513088"
I20260923 23:16:50.071833 6890 raft_consensus.cc:3037] T 79757d715dae41c8ab81c7ff45013b89 P 06385beadc504000bfdbe0c62c0f09c2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:16:50.071900 7158 raft_consensus.cc:3037] T 79757d715dae41c8ab81c7ff45013b89 P 1dbd9c5173d840aca823fc791e513088 [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:16:50.072595 6890 raft_consensus.cc:2454] T 79757d715dae41c8ab81c7ff45013b89 P 06385beadc504000bfdbe0c62c0f09c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19e417d8de4a47d6b511eefbbe7043cf in term 1.
I20260923 23:16:50.072790 6960 leader_election.cc:302] T 79757d715dae41c8ab81c7ff45013b89 P 19e417d8de4a47d6b511eefbbe7043cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06385beadc504000bfdbe0c62c0f09c2, 19e417d8de4a47d6b511eefbbe7043cf; no voters:
I20260923 23:16:50.072793 7158 raft_consensus.cc:2454] T 79757d715dae41c8ab81c7ff45013b89 P 1dbd9c5173d840aca823fc791e513088 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19e417d8de4a47d6b511eefbbe7043cf in term 1.
I20260923 23:16:50.072901 7224 raft_consensus.cc:2781] T 79757d715dae41c8ab81c7ff45013b89 P 19e417d8de4a47d6b511eefbbe7043cf [term 1 FOLLOWER]: Leader election won for term 1
I20260923 23:16:50.073051 7224 raft_consensus.cc:686] T 79757d715dae41c8ab81c7ff45013b89 P 19e417d8de4a47d6b511eefbbe7043cf [term 1 LEADER]: Becoming Leader. State: Replica: 19e417d8de4a47d6b511eefbbe7043cf, State: Running, Role: LEADER
I20260923 23:16:50.073150 7224 consensus_queue.cc:237] T 79757d715dae41c8ab81c7ff45013b89 P 19e417d8de4a47d6b511eefbbe7043cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19e417d8de4a47d6b511eefbbe7043cf" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 34463 } } peers { permanent_uuid: "06385beadc504000bfdbe0c62c0f09c2" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 44099 } } peers { permanent_uuid: "1dbd9c5173d840aca823fc791e513088" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 36069 } }
I20260923 23:16:50.073956 6745 catalog_manager.cc:5495] T 79757d715dae41c8ab81c7ff45013b89 P 19e417d8de4a47d6b511eefbbe7043cf reported cstate change: term changed from 0 to 1, leader changed from <none> to 19e417d8de4a47d6b511eefbbe7043cf (127.24.227.194). New cstate: current_term: 1 leader_uuid: "19e417d8de4a47d6b511eefbbe7043cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19e417d8de4a47d6b511eefbbe7043cf" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 34463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06385beadc504000bfdbe0c62c0f09c2" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 44099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1dbd9c5173d840aca823fc791e513088" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 36069 } health_report { overall_health: UNKNOWN } } }
I20260923 23:16:50.104157 7213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:50.101993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37978 (local address 127.24.227.194:34463)
0923 23:16:50.102035 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0923 23:16:50.102037 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:16:50.102062 (+ 25us) server_negotiation.cc:407] Connection header received
0923 23:16:50.102145 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:16:50.102146 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:16:50.102163 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:16:50.102217 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 23:16:50.102321 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:50.102793 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:50.103461 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:50.103636 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:50.103683 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:16:50.103797 (+ 114us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 23:16:50.103948 (+ 151us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 23:16:50.104011 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0923 23:16:50.104039 (+ 28us) server_negotiation.cc:299] Negotiation successful
0923 23:16:50.104055 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 23:16:50.105959 6890 raft_consensus.cc:1261] T 79757d715dae41c8ab81c7ff45013b89 P 06385beadc504000bfdbe0c62c0f09c2 [term 1 FOLLOWER]: Refusing update from remote peer 19e417d8de4a47d6b511eefbbe7043cf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 23:16:50.106006 7158 raft_consensus.cc:1261] T 79757d715dae41c8ab81c7ff45013b89 P 1dbd9c5173d840aca823fc791e513088 [term 1 FOLLOWER]: Refusing update from remote peer 19e417d8de4a47d6b511eefbbe7043cf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 23:16:50.106371 7228 consensus_queue.cc:1035] T 79757d715dae41c8ab81c7ff45013b89 P 19e417d8de4a47d6b511eefbbe7043cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "1dbd9c5173d840aca823fc791e513088" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 36069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 23:16:50.106515 7224 consensus_queue.cc:1035] T 79757d715dae41c8ab81c7ff45013b89 P 19e417d8de4a47d6b511eefbbe7043cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "06385beadc504000bfdbe0c62c0f09c2" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 44099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 23:16:50.108868 7231 mvcc.cc:205] Tried to move back new op lower bound from 7332681359791276032 to 7332681359660134400. Current Snapshot: MvccSnapshot[applied={T|T < 7332681359791276032}]
I20260923 23:16:50.109501 7232 mvcc.cc:205] Tried to move back new op lower bound from 7332681359791276032 to 7332681359660134400. Current Snapshot: MvccSnapshot[applied={T|T < 7332681359791276032}]
I20260923 23:16:50.111135 6745 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41960:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 23:16:50.111224 6745 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41960:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 23:16:50.111971 6745 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1c752c42c9f74c2491720c7a2b81880a: Sending DeleteTablet for 3 replicas of tablet 79757d715dae41c8ab81c7ff45013b89
I20260923 23:16:50.112299 7004 tablet_service.cc:1514] Processing DeleteTablet for tablet 79757d715dae41c8ab81c7ff45013b89 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 23:16:50 UTC) from {username='kudu', principal='kudu/127.24.227.254@KRBTEST.COM'} at 127.0.0.1:37976
I20260923 23:16:50.112313 6870 tablet_service.cc:1514] Processing DeleteTablet for tablet 79757d715dae41c8ab81c7ff45013b89 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 23:16:50 UTC) from {username='kudu', principal='kudu/127.24.227.254@KRBTEST.COM'} at 127.0.0.1:54562
I20260923 23:16:50.112536 7241 tablet_replica.cc:330] T 79757d715dae41c8ab81c7ff45013b89 P 06385beadc504000bfdbe0c62c0f09c2: stopping tablet replica
I20260923 23:16:50.112587 7138 tablet_service.cc:1514] Processing DeleteTablet for tablet 79757d715dae41c8ab81c7ff45013b89 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 23:16:50 UTC) from {username='kudu', principal='kudu/127.24.227.254@KRBTEST.COM'} at 127.0.0.1:43998
I20260923 23:16:50.112643 7241 raft_consensus.cc:2229] T 79757d715dae41c8ab81c7ff45013b89 P 06385beadc504000bfdbe0c62c0f09c2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 23:16:50.112751 7241 raft_consensus.cc:2258] T 79757d715dae41c8ab81c7ff45013b89 P 06385beadc504000bfdbe0c62c0f09c2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 23:16:50.112797 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 6807
I20260923 23:16:50.112811 7243 tablet_replica.cc:330] T 79757d715dae41c8ab81c7ff45013b89 P 1dbd9c5173d840aca823fc791e513088: stopping tablet replica
I20260923 23:16:50.112936 7243 raft_consensus.cc:2229] T 79757d715dae41c8ab81c7ff45013b89 P 1dbd9c5173d840aca823fc791e513088 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 23:16:50.113049 7243 raft_consensus.cc:2258] T 79757d715dae41c8ab81c7ff45013b89 P 1dbd9c5173d840aca823fc791e513088 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 23:16:50.113528 7243 ts_tablet_manager.cc:1905] T 79757d715dae41c8ab81c7ff45013b89 P 1dbd9c5173d840aca823fc791e513088: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 23:16:50.115197 7243 ts_tablet_manager.cc:1918] T 79757d715dae41c8ab81c7ff45013b89 P 1dbd9c5173d840aca823fc791e513088: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 23:16:50.115271 7243 log.cc:1196] T 79757d715dae41c8ab81c7ff45013b89 P 1dbd9c5173d840aca823fc791e513088: Deleting WAL directory at /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-2/wal/wals/79757d715dae41c8ab81c7ff45013b89
I20260923 23:16:50.115574 7243 ts_tablet_manager.cc:1939] T 79757d715dae41c8ab81c7ff45013b89 P 1dbd9c5173d840aca823fc791e513088: Deleting consensus metadata
I20260923 23:16:50.115957 6732 catalog_manager.cc:4841] TS 1dbd9c5173d840aca823fc791e513088 (127.24.227.195:36069): tablet 79757d715dae41c8ab81c7ff45013b89 (table test-table [id=1c20268fa0404d95a6444446f5156370]) successfully deleted
W20260923 23:16:50.118777 6732 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.227.193:44099 (error 108)
I20260923 23:16:50.118956 7242 tablet_replica.cc:330] T 79757d715dae41c8ab81c7ff45013b89 P 19e417d8de4a47d6b511eefbbe7043cf: stopping tablet replica
I20260923 23:16:50.119040 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 6941
I20260923 23:16:50.119058 7242 raft_consensus.cc:2229] T 79757d715dae41c8ab81c7ff45013b89 P 19e417d8de4a47d6b511eefbbe7043cf [term 1 LEADER]: Raft consensus shutting down.
I20260923 23:16:50.119141 7242 raft_consensus.cc:2258] T 79757d715dae41c8ab81c7ff45013b89 P 19e417d8de4a47d6b511eefbbe7043cf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 23:16:50.120486 7212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:50.120353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.193:44099 (local address 127.0.0.1:54574)
0923 23:16:50.120403 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0923 23:16:50.120453 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.193:44099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260923 23:16:50.123246 6732 catalog_manager.cc:4584] TS 06385beadc504000bfdbe0c62c0f09c2 (127.24.227.193:44099): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 79757d715dae41c8ab81c7ff45013b89: Network error: Client connection negotiation failed: client connection to 127.24.227.193:44099: connect: Connection refused (error 111)
I20260923 23:16:50.123369 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 7075
I20260923 23:16:50.123551 7212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:50.123473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.194:34463 (local address 127.0.0.1:37986)
0923 23:16:50.123506 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0923 23:16:50.123525 (+ 19us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.194:34463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260923 23:16:50.127547 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 6713
I20260923 23:16:50.132460 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.254:41135
--webserver_interface=127.24.227.254
--webserver_port=38573
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab
--principal=kudu/127.24.227.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:33523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.227.254:41135
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:16:50.209517 7245 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:16:50.209692 7245 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:16:50.209712 7245 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:16:50.210762 7245 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 23:16:50.210798 7245 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:16:50.210810 7245 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:16:50.210822 7245 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 23:16:50.210834 7245 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 23:16:50.212247 7245 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:33523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.24.227.254:41135
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab
--principal=kudu/127.24.227.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.254:41135
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.227.254
--webserver_port=38573
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7245
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:16:50.212419 7245 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:16:50.212569 7245 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:16:50.220295 7252 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:16:50.220312 7254 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:16:50.220515 7245 server_base.cc:1034] running on GCE node
W20260923 23:16:50.220274 7251 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:16:50.220811 7245 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:16:50.221093 7245 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:16:50.222225 7245 hybrid_clock.cc:648] HybridClock initialized: now 1790205410222205 us; error 30 us; skew 500 ppm
Sep 23 23:16:50 dist-test-slave-754n krb5kdc[6700](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205410, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:16:50.224579 7245 init.cc:377] Logged in from keytab as kudu/127.24.227.254@KRBTEST.COM (short username kudu)
I20260923 23:16:50.225028 7245 webserver.cc:460] Webserver started at http://127.24.227.254:38573/ using document root <none> and password file <none>
I20260923 23:16:50.225241 7245 fs_manager.cc:359] Metadata directory not provided
I20260923 23:16:50.225315 7245 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:16:50.226572 7245 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 23:16:50.227181 7261 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:16:50.227370 7245 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 23:16:50.227461 7245 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/master-0/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/master-0/wal
uuid: "1c752c42c9f74c2491720c7a2b81880a"
format_stamp: "Formatted at 2026-09-23 23:16:49 on dist-test-slave-754n"
server_key: "b38b2c8c90194a9b4f4aa9b792948634"
server_key_iv: "e3b6e93484d1654b9ad3794cfba4b64f"
server_key_version: "encryptionkey@0"
I20260923 23:16:50.227819 7245 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:16:50.249019 7245 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:16:50.251219 7245 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:16:50.251374 7245 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:16:50.255360 7245 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.254:41135
I20260923 23:16:50.255398 7313 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.254:41135 every 8 connection(s)
I20260923 23:16:50.255787 7245 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
I20260923 23:16:50.256716 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 7245
I20260923 23:16:50.257016 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.227.193:44099
--local_ip_for_outbound_sockets=127.24.227.193
--tserver_master_addrs=127.24.227.254:41135
--webserver_port=40821
--webserver_interface=127.24.227.193
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.193.keytab
--principal=kudu/127.24.227.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:33523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 23:16:50.257826 7314 sys_catalog.cc:263] Verifying existing consensus state
I20260923 23:16:50.258701 7314 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c752c42c9f74c2491720c7a2b81880a: Bootstrap starting.
I20260923 23:16:50.266623 7314 log.cc:824] T 00000000000000000000000000000000 P 1c752c42c9f74c2491720c7a2b81880a: Log is configured to *not* fsync() on all Append() calls
I20260923 23:16:50.268574 7314 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c752c42c9f74c2491720c7a2b81880a: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 23:16:50.268846 7314 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c752c42c9f74c2491720c7a2b81880a: Bootstrap complete.
I20260923 23:16:50.271052 7314 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1c752c42c9f74c2491720c7a2b81880a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c752c42c9f74c2491720c7a2b81880a" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41135 } }
I20260923 23:16:50.271260 7314 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1c752c42c9f74c2491720c7a2b81880a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c752c42c9f74c2491720c7a2b81880a, State: Initialized, Role: FOLLOWER
I20260923 23:16:50.271436 7314 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1c752c42c9f74c2491720c7a2b81880a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c752c42c9f74c2491720c7a2b81880a" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41135 } }
I20260923 23:16:50.271490 7314 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1c752c42c9f74c2491720c7a2b81880a [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 23:16:50.271570 7314 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1c752c42c9f74c2491720c7a2b81880a [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 23:16:50.271629 7314 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1c752c42c9f74c2491720c7a2b81880a [term 1 FOLLOWER]: Advancing to term 2
I20260923 23:16:50.272677 7314 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1c752c42c9f74c2491720c7a2b81880a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c752c42c9f74c2491720c7a2b81880a" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41135 } }
I20260923 23:16:50.272759 7314 leader_election.cc:302] T 00000000000000000000000000000000 P 1c752c42c9f74c2491720c7a2b81880a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1c752c42c9f74c2491720c7a2b81880a; no voters:
I20260923 23:16:50.272934 7314 leader_election.cc:288] T 00000000000000000000000000000000 P 1c752c42c9f74c2491720c7a2b81880a [CANDIDATE]: Term 2 election: Requested vote from peers
I20260923 23:16:50.272969 7318 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1c752c42c9f74c2491720c7a2b81880a [term 2 FOLLOWER]: Leader election won for term 2
I20260923 23:16:50.273077 7318 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1c752c42c9f74c2491720c7a2b81880a [term 2 LEADER]: Becoming Leader. State: Replica: 1c752c42c9f74c2491720c7a2b81880a, State: Running, Role: LEADER
I20260923 23:16:50.273192 7314 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1c752c42c9f74c2491720c7a2b81880a [sys.catalog]: configured and running, proceeding with master startup.
I20260923 23:16:50.273216 7318 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1c752c42c9f74c2491720c7a2b81880a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c752c42c9f74c2491720c7a2b81880a" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41135 } }
I20260923 23:16:50.273564 7319 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c752c42c9f74c2491720c7a2b81880a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1c752c42c9f74c2491720c7a2b81880a. Latest consensus state: current_term: 2 leader_uuid: "1c752c42c9f74c2491720c7a2b81880a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c752c42c9f74c2491720c7a2b81880a" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41135 } } }
I20260923 23:16:50.273651 7319 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c752c42c9f74c2491720c7a2b81880a [sys.catalog]: This master's current role is: LEADER
I20260923 23:16:50.273865 7320 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c752c42c9f74c2491720c7a2b81880a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "1c752c42c9f74c2491720c7a2b81880a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c752c42c9f74c2491720c7a2b81880a" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41135 } } }
I20260923 23:16:50.273945 7320 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c752c42c9f74c2491720c7a2b81880a [sys.catalog]: This master's current role is: LEADER
I20260923 23:16:50.274041 7325 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 23:16:50.274796 7325 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 23:16:50.274892 7325 catalog_manager.cc:1259] Loaded cluster ID: 49a5d989716d404a8f46fc88738ceaf5
I20260923 23:16:50.274971 7325 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 23:16:50.275475 7325 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260923 23:16:50.279897 7325 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260923 23:16:50.282558 7325 catalog_manager.cc:1504] Loading token signing keys...
I20260923 23:16:50.282737 7325 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 1c752c42c9f74c2491720c7a2b81880a: Loaded TSK: 0
I20260923 23:16:50.282933 7325 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260923 23:16:50.337189 7316 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:16:50.337363 7316 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:16:50.337383 7316 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:16:50.338443 7316 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:16:50.338480 7316 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:16:50.338505 7316 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.227.193
I20260923 23:16:50.339821 7316 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:33523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.193.keytab
--principal=kudu/127.24.227.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.227.193:44099
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.227.193
--webserver_port=40821
--tserver_master_addrs=127.24.227.254:41135
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7316
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.227.193
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:16:50.340097 7316 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:16:50.340334 7316 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:16:50.352113 7342 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:16:50.352136 7341 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:16:50.352428 7344 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:16:50.352624 7316 server_base.cc:1034] running on GCE node
I20260923 23:16:50.352814 7316 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:16:50.353086 7316 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:16:50.354265 7316 hybrid_clock.cc:648] HybridClock initialized: now 1790205410354246 us; error 34 us; skew 500 ppm
Sep 23 23:16:50 dist-test-slave-754n krb5kdc[6700](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205410, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:16:50.357261 7316 init.cc:377] Logged in from keytab as kudu/127.24.227.193@KRBTEST.COM (short username kudu)
I20260923 23:16:50.357753 7316 webserver.cc:460] Webserver started at http://127.24.227.193:40821/ using document root <none> and password file <none>
I20260923 23:16:50.358013 7316 fs_manager.cc:359] Metadata directory not provided
I20260923 23:16:50.358084 7316 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:16:50.359694 7316 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 23:16:50.360406 7351 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:16:50.360606 7316 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 23:16:50.360688 7316 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-0/wal
uuid: "06385beadc504000bfdbe0c62c0f09c2"
format_stamp: "Formatted at 2026-09-23 23:16:49 on dist-test-slave-754n"
server_key: "888e34ce2a70e59d063c8bbc4b1e42ae"
server_key_iv: "1be0c765f92a697e6dc06472fb31a01b"
server_key_version: "encryptionkey@0"
I20260923 23:16:50.361065 7316 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:16:50.378937 7316 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:16:50.380998 7316 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:16:50.381134 7316 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:16:50.381367 7316 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 23:16:50.381827 7358 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260923 23:16:50.383136 7316 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260923 23:16:50.383193 7316 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260923 23:16:50.383224 7316 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260923 23:16:50.384011 7316 ts_tablet_manager.cc:610] Registered 1 tablets
I20260923 23:16:50.384053 7316 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.001s
I20260923 23:16:50.384109 7358 tablet_bootstrap.cc:492] T 79757d715dae41c8ab81c7ff45013b89 P 06385beadc504000bfdbe0c62c0f09c2: Bootstrap starting.
I20260923 23:16:50.390511 7316 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.193:44099
I20260923 23:16:50.390673 7465 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.193:44099 every 8 connection(s)
I20260923 23:16:50.391009 7316 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-0/data/info.pb
I20260923 23:16:50.392060 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 7316
I20260923 23:16:50.392318 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.227.194:34463
--local_ip_for_outbound_sockets=127.24.227.194
--tserver_master_addrs=127.24.227.254:41135
--webserver_port=40821
--webserver_interface=127.24.227.194
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.194.keytab
--principal=kudu/127.24.227.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:33523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 23:16:50.393793 7358 log.cc:824] T 79757d715dae41c8ab81c7ff45013b89 P 06385beadc504000bfdbe0c62c0f09c2: Log is configured to *not* fsync() on all Append() calls
I20260923 23:16:50.396440 7358 tablet_bootstrap.cc:492] T 79757d715dae41c8ab81c7ff45013b89 P 06385beadc504000bfdbe0c62c0f09c2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 23:16:50.396778 7358 tablet_bootstrap.cc:492] T 79757d715dae41c8ab81c7ff45013b89 P 06385beadc504000bfdbe0c62c0f09c2: Bootstrap complete.
I20260923 23:16:50.398000 7358 ts_tablet_manager.cc:1397] T 79757d715dae41c8ab81c7ff45013b89 P 06385beadc504000bfdbe0c62c0f09c2: Time spent bootstrapping tablet: real 0.014s user 0.005s sys 0.008s
I20260923 23:16:50.399390 7358 raft_consensus.cc:348] T 79757d715dae41c8ab81c7ff45013b89 P 06385beadc504000bfdbe0c62c0f09c2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19e417d8de4a47d6b511eefbbe7043cf" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 34463 } } peers { permanent_uuid: "06385beadc504000bfdbe0c62c0f09c2" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 44099 } } peers { permanent_uuid: "1dbd9c5173d840aca823fc791e513088" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 36069 } }
I20260923 23:16:50.399763 7358 raft_consensus.cc:729] T 79757d715dae41c8ab81c7ff45013b89 P 06385beadc504000bfdbe0c62c0f09c2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06385beadc504000bfdbe0c62c0f09c2, State: Initialized, Role: FOLLOWER
I20260923 23:16:50.400012 7358 consensus_queue.cc:260] T 79757d715dae41c8ab81c7ff45013b89 P 06385beadc504000bfdbe0c62c0f09c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19e417d8de4a47d6b511eefbbe7043cf" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 34463 } } peers { permanent_uuid: "06385beadc504000bfdbe0c62c0f09c2" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 44099 } } peers { permanent_uuid: "1dbd9c5173d840aca823fc791e513088" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 36069 } }
I20260923 23:16:50.400341 7358 ts_tablet_manager.cc:1428] T 79757d715dae41c8ab81c7ff45013b89 P 06385beadc504000bfdbe0c62c0f09c2: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
Sep 23 23:16:50 dist-test-slave-754n krb5kdc[6700](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205410, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.193@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
I20260923 23:16:50.407095 7468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:50.391667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41135 (local address 127.24.227.193:39099)
0923 23:16:50.394176 (+ 2509us) negotiation.cc:107] Waiting for socket to connect
0923 23:16:50.394735 (+ 559us) client_negotiation.cc:174] Beginning negotiation
0923 23:16:50.395345 (+ 610us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:16:50.395951 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:16:50.395955 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:16:50.396328 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:16:50.396600 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:16:50.396606 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:16:50.398406 (+ 1800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:16:50.398407 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:16:50.399776 (+ 1369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:16:50.399780 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:16:50.400646 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:16:50.400647 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:16:50.400770 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:16:50.401361 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:16:50.401371 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:16:50.403323 (+ 1952us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:16:50.405439 (+ 2116us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:16:50.405463 (+ 24us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:16:50.405465 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:16:50.405744 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:16:50.406004 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:16:50.406005 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:16:50.406005 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:16:50.406127 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:16:50.406504 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:16:50.406509 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:16:50.406658 (+ 149us) client_negotiation.cc:769] Sending connection context
0923 23:16:50.406757 (+ 99us) client_negotiation.cc:240] Negotiation successful
0923 23:16:50.406896 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2470,"thread_start_us":77,"threads_started":1}
I20260923 23:16:50.407135 7469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:50.391780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.193:39099 (local address 127.24.227.254:41135)
0923 23:16:50.395269 (+ 3489us) server_negotiation.cc:206] Beginning negotiation
0923 23:16:50.395279 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 23:16:50.395321 (+ 42us) server_negotiation.cc:407] Connection header received
0923 23:16:50.395613 (+ 292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:16:50.395619 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:16:50.395675 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:16:50.395795 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:16:50.396743 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:50.398288 (+ 1545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:50.399898 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:50.400555 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:50.400667 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:16:50.403430 (+ 2763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:16:50.403434 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:16:50.403434 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:16:50.403445 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:16:50.405356 (+ 1911us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:16:50.405825 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:16:50.405828 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:16:50.405828 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:16:50.405936 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:16:50.406188 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:16:50.406189 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:16:50.406189 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:16:50.406438 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:16:50.406514 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0923 23:16:50.406828 (+ 314us) server_negotiation.cc:299] Negotiation successful
0923 23:16:50.406959 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3445,"thread_start_us":123,"threads_started":1}
I20260923 23:16:50.408509 7466 heartbeater.cc:347] Connected to a master server at 127.24.227.254:41135
I20260923 23:16:50.408598 7466 heartbeater.cc:464] Registering TS with master...
I20260923 23:16:50.408777 7466 heartbeater.cc:511] Master 127.24.227.254:41135 requested a full tablet report, sending...
I20260923 23:16:50.409595 7278 ts_manager.cc:194] Registered new tserver with Master: 06385beadc504000bfdbe0c62c0f09c2 (127.24.227.193:44099)
I20260923 23:16:50.410645 7278 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.227.193@KRBTEST.COM'} at 127.24.227.193:39099
I20260923 23:16:50.411542 7466 heartbeater.cc:503] Master 127.24.227.254:41135 was elected leader, sending a full tablet report...
I20260923 23:16:50.415437 7473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:50.410249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.193:44099 (local address 127.0.0.1:54578)
0923 23:16:50.412144 (+ 1895us) negotiation.cc:107] Waiting for socket to connect
0923 23:16:50.412177 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0923 23:16:50.412232 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:16:50.412505 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:16:50.412507 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:16:50.412509 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:16:50.412559 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:16:50.412560 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:16:50.413740 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:16:50.413741 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:16:50.414465 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:16:50.414467 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:16:50.415248 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:16:50.415249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:16:50.415299 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:16:50.415307 (+ 8us) client_negotiation.cc:769] Sending connection context
0923 23:16:50.415351 (+ 44us) client_negotiation.cc:240] Negotiation successful
0923 23:16:50.415384 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1868,"thread_start_us":77,"threads_started":1}
I20260923 23:16:50.415481 7474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:50.410474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54578 (local address 127.24.227.193:44099)
0923 23:16:50.412340 (+ 1866us) server_negotiation.cc:206] Beginning negotiation
0923 23:16:50.412344 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 23:16:50.412351 (+ 7us) server_negotiation.cc:407] Connection header received
0923 23:16:50.412379 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:16:50.412380 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:16:50.412393 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:16:50.412453 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:16:50.412624 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:50.413658 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:50.414540 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:50.415175 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:50.415243 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:16:50.415285 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0923 23:16:50.415372 (+ 87us) server_negotiation.cc:299] Negotiation successful
0923 23:16:50.415406 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1840,"thread_start_us":69,"threads_started":1}
I20260923 23:16:50.415834 7400 tablet_service.cc:1514] Processing DeleteTablet for tablet 79757d715dae41c8ab81c7ff45013b89 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 23:16:50 UTC) from {username='kudu', principal='kudu/127.24.227.254@KRBTEST.COM'} at 127.0.0.1:54578
I20260923 23:16:50.416355 7475 tablet_replica.cc:330] stopping tablet replica
I20260923 23:16:50.416460 7475 raft_consensus.cc:2229] T 79757d715dae41c8ab81c7ff45013b89 P 06385beadc504000bfdbe0c62c0f09c2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 23:16:50.416543 7475 raft_consensus.cc:2258] T 79757d715dae41c8ab81c7ff45013b89 P 06385beadc504000bfdbe0c62c0f09c2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 23:16:50.416806 7475 ts_tablet_manager.cc:1905] T 79757d715dae41c8ab81c7ff45013b89 P 06385beadc504000bfdbe0c62c0f09c2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 23:16:50.418689 7475 ts_tablet_manager.cc:1918] T 79757d715dae41c8ab81c7ff45013b89 P 06385beadc504000bfdbe0c62c0f09c2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 23:16:50.418744 7475 log.cc:1196] T 79757d715dae41c8ab81c7ff45013b89 P 06385beadc504000bfdbe0c62c0f09c2: Deleting WAL directory at /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-0/wal/wals/79757d715dae41c8ab81c7ff45013b89
I20260923 23:16:50.418968 7475 ts_tablet_manager.cc:1939] T 79757d715dae41c8ab81c7ff45013b89 P 06385beadc504000bfdbe0c62c0f09c2: Deleting consensus metadata
I20260923 23:16:50.419373 7265 catalog_manager.cc:4841] TS 06385beadc504000bfdbe0c62c0f09c2 (127.24.227.193:44099): tablet 79757d715dae41c8ab81c7ff45013b89 (table test-table [id=1c20268fa0404d95a6444446f5156370]) successfully deleted
W20260923 23:16:50.476970 7470 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:16:50.477180 7470 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:16:50.477218 7470 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:16:50.478427 7470 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:16:50.478489 7470 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:16:50.478528 7470 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.227.194
I20260923 23:16:50.479909 7470 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:33523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.194.keytab
--principal=kudu/127.24.227.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.227.194:34463
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.227.194
--webserver_port=40821
--tserver_master_addrs=127.24.227.254:41135
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7470
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.227.194
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:16:50.480154 7470 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:16:50.480352 7470 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:16:50.484895 7481 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:16:50.484961 7480 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:16:50.484987 7483 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:16:50.485119 7470 server_base.cc:1034] running on GCE node
I20260923 23:16:50.485321 7470 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:16:50.485560 7470 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:16:50.486712 7470 hybrid_clock.cc:648] HybridClock initialized: now 1790205410486701 us; error 28 us; skew 500 ppm
Sep 23 23:16:50 dist-test-slave-754n krb5kdc[6700](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205410, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:16:50.489002 7470 init.cc:377] Logged in from keytab as kudu/127.24.227.194@KRBTEST.COM (short username kudu)
I20260923 23:16:50.489377 7470 webserver.cc:460] Webserver started at http://127.24.227.194:40821/ using document root <none> and password file <none>
I20260923 23:16:50.489563 7470 fs_manager.cc:359] Metadata directory not provided
I20260923 23:16:50.489632 7470 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:16:50.490809 7470 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 23:16:50.491340 7490 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:16:50.491485 7470 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 23:16:50.491572 7470 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-1/wal
uuid: "19e417d8de4a47d6b511eefbbe7043cf"
format_stamp: "Formatted at 2026-09-23 23:16:49 on dist-test-slave-754n"
server_key: "18f0ee9a2d01d9996329afa39a60c510"
server_key_iv: "f124b9ef428a1cb011648191998b03ab"
server_key_version: "encryptionkey@0"
I20260923 23:16:50.491859 7470 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:16:50.506820 7470 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:16:50.508519 7470 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:16:50.508677 7470 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:16:50.508882 7470 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 23:16:50.509331 7497 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260923 23:16:50.510598 7470 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260923 23:16:50.510649 7470 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.001s user 0.000s sys 0.000s
I20260923 23:16:50.510703 7470 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260923 23:16:50.511258 7470 ts_tablet_manager.cc:610] Registered 1 tablets
I20260923 23:16:50.511307 7470 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260923 23:16:50.511369 7497 tablet_bootstrap.cc:492] T 79757d715dae41c8ab81c7ff45013b89 P 19e417d8de4a47d6b511eefbbe7043cf: Bootstrap starting.
I20260923 23:16:50.517695 7470 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.194:34463
I20260923 23:16:50.517772 7604 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.194:34463 every 8 connection(s)
I20260923 23:16:50.518083 7470 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-1/data/info.pb
I20260923 23:16:50.518399 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 7470
I20260923 23:16:50.518720 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.227.195:36069
--local_ip_for_outbound_sockets=127.24.227.195
--tserver_master_addrs=127.24.227.254:41135
--webserver_port=36787
--webserver_interface=127.24.227.195
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.195.keytab
--principal=kudu/127.24.227.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:33523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 23:16:50.522176 7497 log.cc:824] T 79757d715dae41c8ab81c7ff45013b89 P 19e417d8de4a47d6b511eefbbe7043cf: Log is configured to *not* fsync() on all Append() calls
I20260923 23:16:50.524333 7497 tablet_bootstrap.cc:492] T 79757d715dae41c8ab81c7ff45013b89 P 19e417d8de4a47d6b511eefbbe7043cf: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 23:16:50.524633 7497 tablet_bootstrap.cc:492] T 79757d715dae41c8ab81c7ff45013b89 P 19e417d8de4a47d6b511eefbbe7043cf: Bootstrap complete.
I20260923 23:16:50.525647 7497 ts_tablet_manager.cc:1397] T 79757d715dae41c8ab81c7ff45013b89 P 19e417d8de4a47d6b511eefbbe7043cf: Time spent bootstrapping tablet: real 0.014s user 0.004s sys 0.008s
Sep 23 23:16:50 dist-test-slave-754n krb5kdc[6700](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205410, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.194@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
I20260923 23:16:50.526736 7497 raft_consensus.cc:348] T 79757d715dae41c8ab81c7ff45013b89 P 19e417d8de4a47d6b511eefbbe7043cf [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19e417d8de4a47d6b511eefbbe7043cf" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 34463 } } peers { permanent_uuid: "06385beadc504000bfdbe0c62c0f09c2" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 44099 } } peers { permanent_uuid: "1dbd9c5173d840aca823fc791e513088" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 36069 } }
I20260923 23:16:50.527074 7497 raft_consensus.cc:729] T 79757d715dae41c8ab81c7ff45013b89 P 19e417d8de4a47d6b511eefbbe7043cf [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19e417d8de4a47d6b511eefbbe7043cf, State: Initialized, Role: FOLLOWER
I20260923 23:16:50.527199 7497 consensus_queue.cc:260] T 79757d715dae41c8ab81c7ff45013b89 P 19e417d8de4a47d6b511eefbbe7043cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19e417d8de4a47d6b511eefbbe7043cf" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 34463 } } peers { permanent_uuid: "06385beadc504000bfdbe0c62c0f09c2" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 44099 } } peers { permanent_uuid: "1dbd9c5173d840aca823fc791e513088" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 36069 } }
I20260923 23:16:50.527446 7497 ts_tablet_manager.cc:1428] T 79757d715dae41c8ab81c7ff45013b89 P 19e417d8de4a47d6b511eefbbe7043cf: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260923 23:16:50.530660 7608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:50.520376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41135 (local address 127.24.227.194:48261)
0923 23:16:50.520559 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0923 23:16:50.520605 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0923 23:16:50.521244 (+ 639us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:16:50.521523 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:16:50.521526 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:16:50.521778 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:16:50.521960 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:16:50.521964 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:16:50.522838 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:16:50.522840 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:16:50.524267 (+ 1427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:16:50.524270 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:16:50.524811 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:16:50.524813 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:16:50.524981 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:16:50.525595 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:16:50.525606 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:16:50.527424 (+ 1818us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:16:50.529388 (+ 1964us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:16:50.529394 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:16:50.529395 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:16:50.529663 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:16:50.529868 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:16:50.529869 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:16:50.529869 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:16:50.529968 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:16:50.530247 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:16:50.530251 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:16:50.530347 (+ 96us) client_negotiation.cc:769] Sending connection context
0923 23:16:50.530421 (+ 74us) client_negotiation.cc:240] Negotiation successful
0923 23:16:50.530536 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":92,"threads_started":1}
I20260923 23:16:50.530668 7469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:50.520509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.194:48261 (local address 127.24.227.254:41135)
0923 23:16:50.520558 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0923 23:16:50.520560 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:16:50.520698 (+ 138us) server_negotiation.cc:407] Connection header received
0923 23:16:50.521370 (+ 672us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:16:50.521372 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:16:50.521388 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:16:50.521470 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:16:50.522058 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:50.522723 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:50.524362 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:50.524713 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:50.524786 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:16:50.527549 (+ 2763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:16:50.527550 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:16:50.527551 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:16:50.527559 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:16:50.529312 (+ 1753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:16:50.529735 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:16:50.529736 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:16:50.529736 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:16:50.529799 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:16:50.530032 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:16:50.530033 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:16:50.530033 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:16:50.530178 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:16:50.530254 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0923 23:16:50.530448 (+ 194us) server_negotiation.cc:299] Negotiation successful
0923 23:16:50.530540 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260923 23:16:50.531425 7605 heartbeater.cc:347] Connected to a master server at 127.24.227.254:41135
I20260923 23:16:50.531512 7605 heartbeater.cc:464] Registering TS with master...
I20260923 23:16:50.531670 7605 heartbeater.cc:511] Master 127.24.227.254:41135 requested a full tablet report, sending...
I20260923 23:16:50.532238 7278 ts_manager.cc:194] Registered new tserver with Master: 19e417d8de4a47d6b511eefbbe7043cf (127.24.227.194:34463)
I20260923 23:16:50.532806 7278 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.227.194@KRBTEST.COM'} at 127.24.227.194:48261
I20260923 23:16:50.534096 7605 heartbeater.cc:503] Master 127.24.227.254:41135 was elected leader, sending a full tablet report...
Sep 23 23:16:50 dist-test-slave-754n krb5kdc[6700](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205410, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.254@KRBTEST.COM for kudu/127.24.227.194@KRBTEST.COM
I20260923 23:16:50.539562 7473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:50.532514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.194:34463 (local address 127.0.0.1:37996)
0923 23:16:50.532562 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0923 23:16:50.532572 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 23:16:50.532612 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:16:50.532844 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:16:50.532845 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:16:50.533135 (+ 290us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:16:50.533166 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:16:50.533168 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:16:50.534126 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:16:50.534127 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:16:50.534730 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:16:50.534732 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:16:50.535133 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:16:50.535134 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:16:50.535191 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:16:50.535676 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:16:50.535681 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:16:50.536946 (+ 1265us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:16:50.538495 (+ 1549us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:16:50.538502 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:16:50.538504 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:16:50.538693 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:16:50.538900 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:16:50.538901 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:16:50.538902 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:16:50.538972 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:16:50.539242 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:16:50.539248 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:16:50.539297 (+ 49us) client_negotiation.cc:769] Sending connection context
0923 23:16:50.539380 (+ 83us) client_negotiation.cc:240] Negotiation successful
0923 23:16:50.539472 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260923 23:16:50.539582 7611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:50.532565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37996 (local address 127.24.227.194:34463)
0923 23:16:50.532706 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0923 23:16:50.532710 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 23:16:50.532717 (+ 7us) server_negotiation.cc:407] Connection header received
0923 23:16:50.532751 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:16:50.532753 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:16:50.532763 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:16:50.532831 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:16:50.533259 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:50.534025 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:50.534804 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:50.535062 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:50.535133 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:16:50.537028 (+ 1895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:16:50.537035 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:16:50.537035 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:16:50.537042 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:16:50.538396 (+ 1354us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:16:50.538774 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:16:50.538777 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:16:50.538777 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:16:50.538831 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:16:50.539046 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:16:50.539047 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:16:50.539047 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:16:50.539176 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:16:50.539260 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0923 23:16:50.539412 (+ 152us) server_negotiation.cc:299] Negotiation successful
0923 23:16:50.539503 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":78,"threads_started":1}
I20260923 23:16:50.539942 7539 tablet_service.cc:1514] Processing DeleteTablet for tablet 79757d715dae41c8ab81c7ff45013b89 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 23:16:50 UTC) from {username='kudu', principal='kudu/127.24.227.254@KRBTEST.COM'} at 127.0.0.1:37996
I20260923 23:16:50.540361 7612 tablet_replica.cc:330] stopping tablet replica
I20260923 23:16:50.540457 7612 raft_consensus.cc:2229] T 79757d715dae41c8ab81c7ff45013b89 P 19e417d8de4a47d6b511eefbbe7043cf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 23:16:50.540515 7612 raft_consensus.cc:2258] T 79757d715dae41c8ab81c7ff45013b89 P 19e417d8de4a47d6b511eefbbe7043cf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 23:16:50.540771 7612 ts_tablet_manager.cc:1905] T 79757d715dae41c8ab81c7ff45013b89 P 19e417d8de4a47d6b511eefbbe7043cf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 23:16:50.543099 7612 ts_tablet_manager.cc:1918] T 79757d715dae41c8ab81c7ff45013b89 P 19e417d8de4a47d6b511eefbbe7043cf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 23:16:50.543169 7612 log.cc:1196] T 79757d715dae41c8ab81c7ff45013b89 P 19e417d8de4a47d6b511eefbbe7043cf: Deleting WAL directory at /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-1/wal/wals/79757d715dae41c8ab81c7ff45013b89
I20260923 23:16:50.543392 7612 ts_tablet_manager.cc:1939] T 79757d715dae41c8ab81c7ff45013b89 P 19e417d8de4a47d6b511eefbbe7043cf: Deleting consensus metadata
I20260923 23:16:50.543812 7265 catalog_manager.cc:4841] TS 19e417d8de4a47d6b511eefbbe7043cf (127.24.227.194:34463): tablet 79757d715dae41c8ab81c7ff45013b89 (table test-table [id=1c20268fa0404d95a6444446f5156370]) successfully deleted
W20260923 23:16:50.602326 7607 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:16:50.602504 7607 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:16:50.602537 7607 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:16:50.603607 7607 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:16:50.603645 7607 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:16:50.603694 7607 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.227.195
I20260923 23:16:50.605057 7607 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:33523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.195.keytab
--principal=kudu/127.24.227.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.227.195:36069
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.227.195
--webserver_port=36787
--tserver_master_addrs=127.24.227.254:41135
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7607
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.227.195
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:16:50.605279 7607 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:16:50.605504 7607 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:16:50.610064 7620 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:16:50.610075 7617 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:16:50.610077 7618 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:16:50.610239 7607 server_base.cc:1034] running on GCE node
I20260923 23:16:50.610476 7607 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:16:50.610693 7607 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:16:50.611851 7607 hybrid_clock.cc:648] HybridClock initialized: now 1790205410611836 us; error 27 us; skew 500 ppm
Sep 23 23:16:50 dist-test-slave-754n krb5kdc[6700](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205410, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:16:50.614202 7607 init.cc:377] Logged in from keytab as kudu/127.24.227.195@KRBTEST.COM (short username kudu)
I20260923 23:16:50.614643 7607 webserver.cc:460] Webserver started at http://127.24.227.195:36787/ using document root <none> and password file <none>
I20260923 23:16:50.614851 7607 fs_manager.cc:359] Metadata directory not provided
I20260923 23:16:50.614943 7607 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:16:50.616204 7607 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 23:16:50.616729 7627 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:16:50.616878 7607 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 23:16:50.616963 7607 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-2/wal
uuid: "1dbd9c5173d840aca823fc791e513088"
format_stamp: "Formatted at 2026-09-23 23:16:49 on dist-test-slave-754n"
server_key: "9dfabea4b8a7cc86f59b5eeac6e2275e"
server_key_iv: "ed1e42305954f38553aae7d94e7497b9"
server_key_version: "encryptionkey@0"
I20260923 23:16:50.617238 7607 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:16:50.639690 7607 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:16:50.641899 7607 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:16:50.642050 7607 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:16:50.642247 7607 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 23:16:50.642527 7607 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 23:16:50.642575 7607 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 23:16:50.642613 7607 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 23:16:50.642640 7607 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 23:16:50.648262 7607 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.195:36069
I20260923 23:16:50.648309 7740 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.195:36069 every 8 connection(s)
I20260923 23:16:50.648617 7607 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-2/data/info.pb
Sep 23 23:16:50 dist-test-slave-754n krb5kdc[6700](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205410, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.195@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
I20260923 23:16:50.655225 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 7607
I20260923 23:16:50.657616 7469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:50.649237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.195:36329 (local address 127.24.227.254:41135)
0923 23:16:50.649310 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0923 23:16:50.649313 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 23:16:50.649526 (+ 213us) server_negotiation.cc:407] Connection header received
0923 23:16:50.650009 (+ 483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:16:50.650010 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:16:50.650021 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:16:50.650079 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:16:50.650965 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:50.651441 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:50.652521 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:50.652731 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:50.652814 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:16:50.654685 (+ 1871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:16:50.654686 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:16:50.654687 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:16:50.654694 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:16:50.656298 (+ 1604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:16:50.656702 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:16:50.656703 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:16:50.656703 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:16:50.656756 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:16:50.657022 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:16:50.657022 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:16:50.657022 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:16:50.657115 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:16:50.657162 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0923 23:16:50.657456 (+ 294us) server_negotiation.cc:299] Negotiation successful
0923 23:16:50.657531 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260923 23:16:50.657747 7743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:50.649242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41135 (local address 127.24.227.195:36329)
0923 23:16:50.649415 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0923 23:16:50.649449 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0923 23:16:50.649896 (+ 447us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:16:50.650144 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:16:50.650148 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:16:50.650678 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:16:50.650859 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:16:50.650863 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:16:50.651538 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:16:50.651539 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:16:50.652429 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:16:50.652433 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:16:50.652821 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:16:50.652822 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:16:50.652946 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:16:50.653327 (+ 381us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:16:50.653335 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:16:50.654586 (+ 1251us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:16:50.656371 (+ 1785us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:16:50.656376 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:16:50.656378 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:16:50.656622 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:16:50.656827 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:16:50.656828 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:16:50.656828 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:16:50.656960 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:16:50.657170 (+ 210us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:16:50.657174 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:16:50.657312 (+ 138us) client_negotiation.cc:769] Sending connection context
0923 23:16:50.657415 (+ 103us) client_negotiation.cc:240] Negotiation successful
0923 23:16:50.657525 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":101,"threads_started":1}
I20260923 23:16:50.658195 7741 heartbeater.cc:347] Connected to a master server at 127.24.227.254:41135
I20260923 23:16:50.658289 7741 heartbeater.cc:464] Registering TS with master...
I20260923 23:16:50.658459 7741 heartbeater.cc:511] Master 127.24.227.254:41135 requested a full tablet report, sending...
I20260923 23:16:50.658839 7278 ts_manager.cc:194] Registered new tserver with Master: 1dbd9c5173d840aca823fc791e513088 (127.24.227.195:36069)
I20260923 23:16:50.659158 7278 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.227.195@KRBTEST.COM'} at 127.24.227.195:36329
I20260923 23:16:50.661932 7745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:50.655607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41964 (local address 127.24.227.254:41135)
0923 23:16:50.655699 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0923 23:16:50.655701 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:16:50.655734 (+ 33us) server_negotiation.cc:407] Connection header received
0923 23:16:50.655811 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:16:50.655812 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:16:50.655820 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:16:50.655920 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:16:50.656582 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:50.657297 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:50.658330 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:50.658582 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:50.658660 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:16:50.660020 (+ 1360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:16:50.660021 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:16:50.660021 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:16:50.660027 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:16:50.661121 (+ 1094us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:16:50.661398 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:16:50.661399 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:16:50.661399 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:16:50.661452 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:16:50.661562 (+ 110us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:16:50.661562 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:16:50.661563 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:16:50.661639 (+ 76us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:16:50.661744 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0923 23:16:50.661775 (+ 31us) server_negotiation.cc:299] Negotiation successful
0923 23:16:50.661831 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75,"thread_start_us":53,"threads_started":1}
I20260923 23:16:50.662503 25487 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 23:16:50.664992 7745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:50.662790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41972 (local address 127.24.227.254:41135)
0923 23:16:50.662838 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0923 23:16:50.662840 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:16:50.662942 (+ 102us) server_negotiation.cc:407] Connection header received
0923 23:16:50.662987 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:16:50.662988 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:16:50.663011 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:16:50.663069 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 23:16:50.663162 (+ 93us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:50.663577 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:50.664285 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:50.664479 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:50.664522 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:16:50.664678 (+ 156us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 23:16:50.664766 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 23:16:50.664819 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0923 23:16:50.664852 (+ 33us) server_negotiation.cc:299] Negotiation successful
0923 23:16:50.664868 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260923 23:16:50.665206 7278 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:41972:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 23:16:50.666723 7278 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 23:16:50.668676 7539 tablet_service.cc:1467] Processing CreateTablet for tablet 50ea81924f5242d3808257c752f058cb (DEFAULT_TABLE table=test-table [id=c371d97601e547a1a9644c96b7bd7721]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 23:16:50.668717 7400 tablet_service.cc:1467] Processing CreateTablet for tablet 50ea81924f5242d3808257c752f058cb (DEFAULT_TABLE table=test-table [id=c371d97601e547a1a9644c96b7bd7721]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 23:16:50.668865 7539 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50ea81924f5242d3808257c752f058cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:16:50.668892 7400 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50ea81924f5242d3808257c752f058cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:16:50.670179 7497 tablet_bootstrap.cc:492] T 50ea81924f5242d3808257c752f058cb P 19e417d8de4a47d6b511eefbbe7043cf: Bootstrap starting.
I20260923 23:16:50.670429 7358 tablet_bootstrap.cc:492] T 50ea81924f5242d3808257c752f058cb P 06385beadc504000bfdbe0c62c0f09c2: Bootstrap starting.
I20260923 23:16:50.670580 7497 tablet_bootstrap.cc:654] T 50ea81924f5242d3808257c752f058cb P 19e417d8de4a47d6b511eefbbe7043cf: Neither blocks nor log segments found. Creating new log.
I20260923 23:16:50.670739 7358 tablet_bootstrap.cc:654] T 50ea81924f5242d3808257c752f058cb P 06385beadc504000bfdbe0c62c0f09c2: Neither blocks nor log segments found. Creating new log.
I20260923 23:16:50.671084 7497 tablet_bootstrap.cc:492] T 50ea81924f5242d3808257c752f058cb P 19e417d8de4a47d6b511eefbbe7043cf: No bootstrap required, opened a new log
I20260923 23:16:50.671136 7497 ts_tablet_manager.cc:1397] T 50ea81924f5242d3808257c752f058cb P 19e417d8de4a47d6b511eefbbe7043cf: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260923 23:16:50.671216 7358 tablet_bootstrap.cc:492] T 50ea81924f5242d3808257c752f058cb P 06385beadc504000bfdbe0c62c0f09c2: No bootstrap required, opened a new log
I20260923 23:16:50.671264 7358 ts_tablet_manager.cc:1397] T 50ea81924f5242d3808257c752f058cb P 06385beadc504000bfdbe0c62c0f09c2: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260923 23:16:50.671370 7358 raft_consensus.cc:348] T 50ea81924f5242d3808257c752f058cb P 06385beadc504000bfdbe0c62c0f09c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbd9c5173d840aca823fc791e513088" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 36069 } } peers { permanent_uuid: "06385beadc504000bfdbe0c62c0f09c2" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 44099 } } peers { permanent_uuid: "19e417d8de4a47d6b511eefbbe7043cf" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 34463 } }
I20260923 23:16:50.671449 7358 raft_consensus.cc:374] T 50ea81924f5242d3808257c752f058cb P 06385beadc504000bfdbe0c62c0f09c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:16:50.671499 7358 raft_consensus.cc:729] T 50ea81924f5242d3808257c752f058cb P 06385beadc504000bfdbe0c62c0f09c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06385beadc504000bfdbe0c62c0f09c2, State: Initialized, Role: FOLLOWER
I20260923 23:16:50.671489 7497 raft_consensus.cc:348] T 50ea81924f5242d3808257c752f058cb P 19e417d8de4a47d6b511eefbbe7043cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbd9c5173d840aca823fc791e513088" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 36069 } } peers { permanent_uuid: "06385beadc504000bfdbe0c62c0f09c2" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 44099 } } peers { permanent_uuid: "19e417d8de4a47d6b511eefbbe7043cf" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 34463 } }
I20260923 23:16:50.671559 7497 raft_consensus.cc:374] T 50ea81924f5242d3808257c752f058cb P 19e417d8de4a47d6b511eefbbe7043cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:16:50.671545 7358 consensus_queue.cc:260] T 50ea81924f5242d3808257c752f058cb P 06385beadc504000bfdbe0c62c0f09c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbd9c5173d840aca823fc791e513088" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 36069 } } peers { permanent_uuid: "06385beadc504000bfdbe0c62c0f09c2" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 44099 } } peers { permanent_uuid: "19e417d8de4a47d6b511eefbbe7043cf" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 34463 } }
I20260923 23:16:50.671597 7497 raft_consensus.cc:729] T 50ea81924f5242d3808257c752f058cb P 19e417d8de4a47d6b511eefbbe7043cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19e417d8de4a47d6b511eefbbe7043cf, State: Initialized, Role: FOLLOWER
I20260923 23:16:50.671643 7358 ts_tablet_manager.cc:1428] T 50ea81924f5242d3808257c752f058cb P 06385beadc504000bfdbe0c62c0f09c2: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 23:16:50.671640 7497 consensus_queue.cc:260] T 50ea81924f5242d3808257c752f058cb P 19e417d8de4a47d6b511eefbbe7043cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbd9c5173d840aca823fc791e513088" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 36069 } } peers { permanent_uuid: "06385beadc504000bfdbe0c62c0f09c2" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 44099 } } peers { permanent_uuid: "19e417d8de4a47d6b511eefbbe7043cf" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 34463 } }
I20260923 23:16:50.671729 7497 ts_tablet_manager.cc:1428] T 50ea81924f5242d3808257c752f058cb P 19e417d8de4a47d6b511eefbbe7043cf: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260923 23:16:50.672299 7747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:50.668410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44008 (local address 127.24.227.195:36069)
0923 23:16:50.668567 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0923 23:16:50.668572 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 23:16:50.668581 (+ 9us) server_negotiation.cc:407] Connection header received
0923 23:16:50.668613 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:16:50.668615 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:16:50.668628 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:16:50.668854 (+ 226us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:16:50.668907 (+ 53us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:50.669980 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:50.671255 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:50.672018 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:50.672080 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:16:50.672122 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0923 23:16:50.672220 (+ 98us) server_negotiation.cc:299] Negotiation successful
0923 23:16:50.672246 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":87,"threads_started":1}
I20260923 23:16:50.672303 7473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:50.668335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.195:36069 (local address 127.0.0.1:44008)
0923 23:16:50.668379 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0923 23:16:50.668389 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 23:16:50.668424 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:16:50.668737 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:16:50.668738 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:16:50.668740 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:16:50.668787 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:16:50.668788 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:16:50.670101 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:16:50.670102 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:16:50.671149 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:16:50.671152 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:16:50.672079 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:16:50.672080 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:16:50.672145 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:16:50.672148 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 23:16:50.672212 (+ 64us) client_negotiation.cc:240] Negotiation successful
0923 23:16:50.672245 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260923 23:16:50.672886 7675 tablet_service.cc:1467] Processing CreateTablet for tablet 50ea81924f5242d3808257c752f058cb (DEFAULT_TABLE table=test-table [id=c371d97601e547a1a9644c96b7bd7721]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 23:16:50.673193 7675 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50ea81924f5242d3808257c752f058cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:16:50.675774 7750 tablet_bootstrap.cc:492] T 50ea81924f5242d3808257c752f058cb P 1dbd9c5173d840aca823fc791e513088: Bootstrap starting.
I20260923 23:16:50.676314 7750 tablet_bootstrap.cc:654] T 50ea81924f5242d3808257c752f058cb P 1dbd9c5173d840aca823fc791e513088: Neither blocks nor log segments found. Creating new log.
I20260923 23:16:50.676849 7750 log.cc:824] T 50ea81924f5242d3808257c752f058cb P 1dbd9c5173d840aca823fc791e513088: Log is configured to *not* fsync() on all Append() calls
I20260923 23:16:50.677465 7750 tablet_bootstrap.cc:492] T 50ea81924f5242d3808257c752f058cb P 1dbd9c5173d840aca823fc791e513088: No bootstrap required, opened a new log
I20260923 23:16:50.677515 7750 ts_tablet_manager.cc:1397] T 50ea81924f5242d3808257c752f058cb P 1dbd9c5173d840aca823fc791e513088: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 23:16:50.678747 7750 raft_consensus.cc:348] T 50ea81924f5242d3808257c752f058cb P 1dbd9c5173d840aca823fc791e513088 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbd9c5173d840aca823fc791e513088" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 36069 } } peers { permanent_uuid: "06385beadc504000bfdbe0c62c0f09c2" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 44099 } } peers { permanent_uuid: "19e417d8de4a47d6b511eefbbe7043cf" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 34463 } }
I20260923 23:16:50.678864 7750 raft_consensus.cc:374] T 50ea81924f5242d3808257c752f058cb P 1dbd9c5173d840aca823fc791e513088 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:16:50.678903 7750 raft_consensus.cc:729] T 50ea81924f5242d3808257c752f058cb P 1dbd9c5173d840aca823fc791e513088 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dbd9c5173d840aca823fc791e513088, State: Initialized, Role: FOLLOWER
I20260923 23:16:50.679014 7750 consensus_queue.cc:260] T 50ea81924f5242d3808257c752f058cb P 1dbd9c5173d840aca823fc791e513088 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbd9c5173d840aca823fc791e513088" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 36069 } } peers { permanent_uuid: "06385beadc504000bfdbe0c62c0f09c2" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 44099 } } peers { permanent_uuid: "19e417d8de4a47d6b511eefbbe7043cf" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 34463 } }
I20260923 23:16:50.679244 7750 ts_tablet_manager.cc:1428] T 50ea81924f5242d3808257c752f058cb P 1dbd9c5173d840aca823fc791e513088: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260923 23:16:50.679257 7741 heartbeater.cc:503] Master 127.24.227.254:41135 was elected leader, sending a full tablet report...
W20260923 23:16:50.768579 7606 tablet.cc:2369] T 50ea81924f5242d3808257c752f058cb P 19e417d8de4a47d6b511eefbbe7043cf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 23:16:50.881731 7752 raft_consensus.cc:482] T 50ea81924f5242d3808257c752f058cb P 1dbd9c5173d840aca823fc791e513088 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 23:16:50.881892 7752 raft_consensus.cc:504] T 50ea81924f5242d3808257c752f058cb P 1dbd9c5173d840aca823fc791e513088 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbd9c5173d840aca823fc791e513088" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 36069 } } peers { permanent_uuid: "06385beadc504000bfdbe0c62c0f09c2" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 44099 } } peers { permanent_uuid: "19e417d8de4a47d6b511eefbbe7043cf" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 34463 } }
I20260923 23:16:50.882300 7752 leader_election.cc:288] T 50ea81924f5242d3808257c752f058cb P 1dbd9c5173d840aca823fc791e513088 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06385beadc504000bfdbe0c62c0f09c2 (127.24.227.193:44099), 19e417d8de4a47d6b511eefbbe7043cf (127.24.227.194:34463)
I20260923 23:16:50.885124 7474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:50.882528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.195:56735 (local address 127.24.227.193:44099)
0923 23:16:50.882567 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0923 23:16:50.882569 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:16:50.882582 (+ 13us) server_negotiation.cc:407] Connection header received
0923 23:16:50.882725 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:16:50.882726 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:16:50.882737 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:16:50.882829 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:16:50.882991 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:50.883460 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:50.884436 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:50.884804 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:50.884882 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:16:50.884891 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0923 23:16:50.885034 (+ 143us) server_negotiation.cc:299] Negotiation successful
0923 23:16:50.885067 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260923 23:16:50.885124 7743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:50.882480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.193:44099 (local address 127.24.227.195:56735)
0923 23:16:50.882529 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0923 23:16:50.882541 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 23:16:50.882582 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:16:50.882836 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:16:50.882837 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:16:50.882839 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:16:50.882909 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:16:50.882911 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:16:50.883536 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:16:50.883537 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:16:50.884352 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:16:50.884354 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:16:50.884880 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:16:50.884881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:16:50.884958 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:16:50.884961 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 23:16:50.885017 (+ 56us) client_negotiation.cc:240] Negotiation successful
0923 23:16:50.885056 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27,"mutex_wait_us":5}
I20260923 23:16:50.885397 7611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:50.882676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.195:38569 (local address 127.24.227.194:34463)
0923 23:16:50.882714 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0923 23:16:50.882716 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:16:50.882859 (+ 143us) server_negotiation.cc:407] Connection header received
0923 23:16:50.882934 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:16:50.882935 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:16:50.882963 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:16:50.883042 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:16:50.883200 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:50.883684 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:50.884640 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:50.885022 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:50.885115 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:16:50.885191 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0923 23:16:50.885291 (+ 100us) server_negotiation.cc:299] Negotiation successful
0923 23:16:50.885342 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260923 23:16:50.885398 7753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:50.882602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.194:34463 (local address 127.24.227.195:38569)
0923 23:16:50.882788 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0923 23:16:50.882803 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0923 23:16:50.882869 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:16:50.883044 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:16:50.883045 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:16:50.883047 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:16:50.883122 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:16:50.883123 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:16:50.883762 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:16:50.883763 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:16:50.884546 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:16:50.884548 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:16:50.885118 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:16:50.885119 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:16:50.885184 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:16:50.885187 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 23:16:50.885273 (+ 86us) client_negotiation.cc:240] Negotiation successful
0923 23:16:50.885315 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":142,"threads_started":1}
I20260923 23:16:50.885455 7420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50ea81924f5242d3808257c752f058cb" candidate_uuid: "1dbd9c5173d840aca823fc791e513088" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06385beadc504000bfdbe0c62c0f09c2" is_pre_election: true
I20260923 23:16:50.885627 7420 raft_consensus.cc:2454] T 50ea81924f5242d3808257c752f058cb P 06385beadc504000bfdbe0c62c0f09c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1dbd9c5173d840aca823fc791e513088 in term 0.
I20260923 23:16:50.885854 7631 leader_election.cc:302] T 50ea81924f5242d3808257c752f058cb P 1dbd9c5173d840aca823fc791e513088 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06385beadc504000bfdbe0c62c0f09c2, 1dbd9c5173d840aca823fc791e513088; no voters:
I20260923 23:16:50.885987 7752 raft_consensus.cc:2781] T 50ea81924f5242d3808257c752f058cb P 1dbd9c5173d840aca823fc791e513088 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 23:16:50.885973 7559 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50ea81924f5242d3808257c752f058cb" candidate_uuid: "1dbd9c5173d840aca823fc791e513088" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19e417d8de4a47d6b511eefbbe7043cf" is_pre_election: true
I20260923 23:16:50.886052 7752 raft_consensus.cc:482] T 50ea81924f5242d3808257c752f058cb P 1dbd9c5173d840aca823fc791e513088 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 23:16:50.886076 7752 raft_consensus.cc:3037] T 50ea81924f5242d3808257c752f058cb P 1dbd9c5173d840aca823fc791e513088 [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:16:50.886117 7559 raft_consensus.cc:2454] T 50ea81924f5242d3808257c752f058cb P 19e417d8de4a47d6b511eefbbe7043cf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1dbd9c5173d840aca823fc791e513088 in term 0.
I20260923 23:16:50.886770 7752 raft_consensus.cc:504] T 50ea81924f5242d3808257c752f058cb P 1dbd9c5173d840aca823fc791e513088 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbd9c5173d840aca823fc791e513088" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 36069 } } peers { permanent_uuid: "06385beadc504000bfdbe0c62c0f09c2" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 44099 } } peers { permanent_uuid: "19e417d8de4a47d6b511eefbbe7043cf" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 34463 } }
I20260923 23:16:50.886881 7752 leader_election.cc:288] T 50ea81924f5242d3808257c752f058cb P 1dbd9c5173d840aca823fc791e513088 [CANDIDATE]: Term 1 election: Requested vote from peers 06385beadc504000bfdbe0c62c0f09c2 (127.24.227.193:44099), 19e417d8de4a47d6b511eefbbe7043cf (127.24.227.194:34463)
I20260923 23:16:50.887022 7420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50ea81924f5242d3808257c752f058cb" candidate_uuid: "1dbd9c5173d840aca823fc791e513088" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06385beadc504000bfdbe0c62c0f09c2"
I20260923 23:16:50.887063 7559 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50ea81924f5242d3808257c752f058cb" candidate_uuid: "1dbd9c5173d840aca823fc791e513088" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19e417d8de4a47d6b511eefbbe7043cf"
I20260923 23:16:50.887116 7420 raft_consensus.cc:3037] T 50ea81924f5242d3808257c752f058cb P 06385beadc504000bfdbe0c62c0f09c2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:16:50.887118 7559 raft_consensus.cc:3037] T 50ea81924f5242d3808257c752f058cb P 19e417d8de4a47d6b511eefbbe7043cf [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:16:50.887789 7559 raft_consensus.cc:2454] T 50ea81924f5242d3808257c752f058cb P 19e417d8de4a47d6b511eefbbe7043cf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1dbd9c5173d840aca823fc791e513088 in term 1.
I20260923 23:16:50.887996 7631 leader_election.cc:302] T 50ea81924f5242d3808257c752f058cb P 1dbd9c5173d840aca823fc791e513088 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19e417d8de4a47d6b511eefbbe7043cf, 1dbd9c5173d840aca823fc791e513088; no voters:
I20260923 23:16:50.888079 7752 raft_consensus.cc:2781] T 50ea81924f5242d3808257c752f058cb P 1dbd9c5173d840aca823fc791e513088 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 23:16:50.888217 7752 raft_consensus.cc:686] T 50ea81924f5242d3808257c752f058cb P 1dbd9c5173d840aca823fc791e513088 [term 1 LEADER]: Becoming Leader. State: Replica: 1dbd9c5173d840aca823fc791e513088, State: Running, Role: LEADER
I20260923 23:16:50.888307 7752 consensus_queue.cc:237] T 50ea81924f5242d3808257c752f058cb P 1dbd9c5173d840aca823fc791e513088 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbd9c5173d840aca823fc791e513088" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 36069 } } peers { permanent_uuid: "06385beadc504000bfdbe0c62c0f09c2" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 44099 } } peers { permanent_uuid: "19e417d8de4a47d6b511eefbbe7043cf" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 34463 } }
I20260923 23:16:50.888332 7420 raft_consensus.cc:2454] T 50ea81924f5242d3808257c752f058cb P 06385beadc504000bfdbe0c62c0f09c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1dbd9c5173d840aca823fc791e513088 in term 1.
I20260923 23:16:50.889155 7277 catalog_manager.cc:5495] T 50ea81924f5242d3808257c752f058cb P 1dbd9c5173d840aca823fc791e513088 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1dbd9c5173d840aca823fc791e513088 (127.24.227.195). New cstate: current_term: 1 leader_uuid: "1dbd9c5173d840aca823fc791e513088" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbd9c5173d840aca823fc791e513088" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 36069 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06385beadc504000bfdbe0c62c0f09c2" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 44099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "19e417d8de4a47d6b511eefbbe7043cf" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 34463 } health_report { overall_health: UNKNOWN } } }
W20260923 23:16:50.894400 7467 tablet.cc:2369] T 50ea81924f5242d3808257c752f058cb P 06385beadc504000bfdbe0c62c0f09c2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 23:16:50.897123 7747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:50.895140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44012 (local address 127.24.227.195:36069)
0923 23:16:50.895175 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0923 23:16:50.895177 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:16:50.895186 (+ 9us) server_negotiation.cc:407] Connection header received
0923 23:16:50.895230 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:16:50.895231 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:16:50.895248 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:16:50.895328 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 23:16:50.895405 (+ 77us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:50.895830 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:50.896518 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:50.896709 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:50.896753 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:16:50.896887 (+ 134us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 23:16:50.896973 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 23:16:50.897025 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0923 23:16:50.897054 (+ 29us) server_negotiation.cc:299] Negotiation successful
0923 23:16:50.897067 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260923 23:16:50.898669 7420 raft_consensus.cc:1261] T 50ea81924f5242d3808257c752f058cb P 06385beadc504000bfdbe0c62c0f09c2 [term 1 FOLLOWER]: Refusing update from remote peer 1dbd9c5173d840aca823fc791e513088: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 23:16:50.898916 7559 raft_consensus.cc:1261] T 50ea81924f5242d3808257c752f058cb P 19e417d8de4a47d6b511eefbbe7043cf [term 1 FOLLOWER]: Refusing update from remote peer 1dbd9c5173d840aca823fc791e513088: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 23:16:50.898972 7754 consensus_queue.cc:1035] T 50ea81924f5242d3808257c752f058cb P 1dbd9c5173d840aca823fc791e513088 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06385beadc504000bfdbe0c62c0f09c2" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 44099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260923 23:16:50.899056 7742 tablet.cc:2369] T 50ea81924f5242d3808257c752f058cb P 1dbd9c5173d840aca823fc791e513088: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 23:16:50.899138 7752 consensus_queue.cc:1035] T 50ea81924f5242d3808257c752f058cb P 1dbd9c5173d840aca823fc791e513088 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19e417d8de4a47d6b511eefbbe7043cf" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 34463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 23:16:50.901398 7758 mvcc.cc:205] Tried to move back new op lower bound from 7332681363038478336 to 7332681362999259136. Current Snapshot: MvccSnapshot[applied={T|T < 7332681363038478336 or (T in {7332681363038478336})}]
I20260923 23:16:50.903036 7278 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:41972:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 23:16:50.903115 7278 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:41972:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 23:16:50.903769 7278 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1c752c42c9f74c2491720c7a2b81880a: Sending DeleteTablet for 3 replicas of tablet 50ea81924f5242d3808257c752f058cb
I20260923 23:16:50.904100 7400 tablet_service.cc:1514] Processing DeleteTablet for tablet 50ea81924f5242d3808257c752f058cb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 23:16:50 UTC) from {username='kudu', principal='kudu/127.24.227.254@KRBTEST.COM'} at 127.0.0.1:54578
I20260923 23:16:50.904088 7675 tablet_service.cc:1514] Processing DeleteTablet for tablet 50ea81924f5242d3808257c752f058cb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 23:16:50 UTC) from {username='kudu', principal='kudu/127.24.227.254@KRBTEST.COM'} at 127.0.0.1:44008
I20260923 23:16:50.904177 7539 tablet_service.cc:1514] Processing DeleteTablet for tablet 50ea81924f5242d3808257c752f058cb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 23:16:50 UTC) from {username='kudu', principal='kudu/127.24.227.254@KRBTEST.COM'} at 127.0.0.1:37996
I20260923 23:16:50.904199 7475 tablet_replica.cc:330] T 50ea81924f5242d3808257c752f058cb P 06385beadc504000bfdbe0c62c0f09c2: stopping tablet replica
I20260923 23:16:50.904235 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 7316
I20260923 23:16:50.904247 7612 tablet_replica.cc:330] T 50ea81924f5242d3808257c752f058cb P 19e417d8de4a47d6b511eefbbe7043cf: stopping tablet replica
I20260923 23:16:50.904254 7475 raft_consensus.cc:2229] T 50ea81924f5242d3808257c752f058cb P 06385beadc504000bfdbe0c62c0f09c2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 23:16:50.904292 7612 raft_consensus.cc:2229] T 50ea81924f5242d3808257c752f058cb P 19e417d8de4a47d6b511eefbbe7043cf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 23:16:50.904304 7475 raft_consensus.cc:2258] T 50ea81924f5242d3808257c752f058cb P 06385beadc504000bfdbe0c62c0f09c2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 23:16:50.904333 7612 raft_consensus.cc:2258] T 50ea81924f5242d3808257c752f058cb P 19e417d8de4a47d6b511eefbbe7043cf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 23:16:50.904946 7612 ts_tablet_manager.cc:1905] T 50ea81924f5242d3808257c752f058cb P 19e417d8de4a47d6b511eefbbe7043cf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 23:16:50.906394 7612 ts_tablet_manager.cc:1918] T 50ea81924f5242d3808257c752f058cb P 19e417d8de4a47d6b511eefbbe7043cf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 23:16:50.906453 7612 log.cc:1196] T 50ea81924f5242d3808257c752f058cb P 19e417d8de4a47d6b511eefbbe7043cf: Deleting WAL directory at /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-1/wal/wals/50ea81924f5242d3808257c752f058cb
I20260923 23:16:50.906674 7612 ts_tablet_manager.cc:1939] T 50ea81924f5242d3808257c752f058cb P 19e417d8de4a47d6b511eefbbe7043cf: Deleting consensus metadata
I20260923 23:16:50.907140 7265 catalog_manager.cc:4841] TS 19e417d8de4a47d6b511eefbbe7043cf (127.24.227.194:34463): tablet 50ea81924f5242d3808257c752f058cb (table test-table [id=c371d97601e547a1a9644c96b7bd7721]) successfully deleted
I20260923 23:16:50.907979 7767 tablet_replica.cc:330] T 50ea81924f5242d3808257c752f058cb P 1dbd9c5173d840aca823fc791e513088: stopping tablet replica
I20260923 23:16:50.908080 7767 raft_consensus.cc:2229] T 50ea81924f5242d3808257c752f058cb P 1dbd9c5173d840aca823fc791e513088 [term 1 LEADER]: Raft consensus shutting down.
I20260923 23:16:50.908167 7767 raft_consensus.cc:2258] T 50ea81924f5242d3808257c752f058cb P 1dbd9c5173d840aca823fc791e513088 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 23:16:50.908522 7767 ts_tablet_manager.cc:1905] T 50ea81924f5242d3808257c752f058cb P 1dbd9c5173d840aca823fc791e513088: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 23:16:50.910585 7767 ts_tablet_manager.cc:1918] T 50ea81924f5242d3808257c752f058cb P 1dbd9c5173d840aca823fc791e513088: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 23:16:50.910660 7767 log.cc:1196] T 50ea81924f5242d3808257c752f058cb P 1dbd9c5173d840aca823fc791e513088: Deleting WAL directory at /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-2/wal/wals/50ea81924f5242d3808257c752f058cb
W20260923 23:16:50.910820 7265 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.227.193:44099 (error 108)
I20260923 23:16:50.910936 7767 ts_tablet_manager.cc:1939] T 50ea81924f5242d3808257c752f058cb P 1dbd9c5173d840aca823fc791e513088: Deleting consensus metadata
I20260923 23:16:50.911294 7265 catalog_manager.cc:4841] TS 1dbd9c5173d840aca823fc791e513088 (127.24.227.195:36069): tablet 50ea81924f5242d3808257c752f058cb (table test-table [id=c371d97601e547a1a9644c96b7bd7721]) successfully deleted
W20260923 23:16:50.911630 7473 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 23:16:50.911244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.193:44099 (local address 127.0.0.1:54586)
0923 23:16:50.911306 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0923 23:16:50.911318 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 23:16:50.911364 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:16:50.911587 (+ 223us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.193:44099: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":43}
W20260923 23:16:50.911748 7265 catalog_manager.cc:4584] TS 06385beadc504000bfdbe0c62c0f09c2 (127.24.227.193:44099): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 50ea81924f5242d3808257c752f058cb: Network error: Client connection negotiation failed: client connection to 127.24.227.193:44099: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260923 23:16:50.912300 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 7470
I20260923 23:16:50.916493 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 7607
I20260923 23:16:50.920506 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 7245
I20260923 23:16:50.925698 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.254:41135
--webserver_interface=127.24.227.254
--webserver_port=38573
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab
--principal=kudu/127.24.227.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:33523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.227.254:41135
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:16:51.002305 7769 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:16:51.002511 7769 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:16:51.002532 7769 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:16:51.003649 7769 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 23:16:51.003691 7769 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:16:51.003705 7769 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:16:51.003750 7769 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 23:16:51.003767 7769 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 23:16:51.005154 7769 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:33523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.227.254:41135
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab
--principal=kudu/127.24.227.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.254:41135
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.227.254
--webserver_port=38573
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7769
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:16:51.005380 7769 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:16:51.005604 7769 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260923 23:16:51.010275 7769 server_base.cc:1034] running on GCE node
W20260923 23:16:51.010228 7777 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:16:51.010221 7774 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:16:51.010234 7775 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:16:51.010617 7769 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:16:51.010883 7769 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:16:51.012039 7769 hybrid_clock.cc:648] HybridClock initialized: now 1790205411012034 us; error 34 us; skew 500 ppm
Sep 23 23:16:51 dist-test-slave-754n krb5kdc[6700](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205411, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:16:51.014285 7769 init.cc:377] Logged in from keytab as kudu/127.24.227.254@KRBTEST.COM (short username kudu)
I20260923 23:16:51.014674 7769 webserver.cc:460] Webserver started at http://127.24.227.254:38573/ using document root <none> and password file <none>
I20260923 23:16:51.014897 7769 fs_manager.cc:359] Metadata directory not provided
I20260923 23:16:51.014966 7769 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:16:51.016215 7769 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 23:16:51.016794 7784 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:16:51.016973 7769 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 23:16:51.017063 7769 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/master-0/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/master-0/wal
uuid: "1c752c42c9f74c2491720c7a2b81880a"
format_stamp: "Formatted at 2026-09-23 23:16:49 on dist-test-slave-754n"
server_key: "b38b2c8c90194a9b4f4aa9b792948634"
server_key_iv: "e3b6e93484d1654b9ad3794cfba4b64f"
server_key_version: "encryptionkey@0"
I20260923 23:16:51.017344 7769 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:16:51.025233 7769 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:16:51.026839 7769 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:16:51.026980 7769 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:16:51.030622 7769 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.254:41135
I20260923 23:16:51.030649 7836 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.254:41135 every 8 connection(s)
I20260923 23:16:51.031004 7769 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
I20260923 23:16:51.032490 7837 sys_catalog.cc:263] Verifying existing consensus state
I20260923 23:16:51.033035 7837 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c752c42c9f74c2491720c7a2b81880a: Bootstrap starting.
I20260923 23:16:51.040041 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 7769
I20260923 23:16:51.040377 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.227.193:44099
--local_ip_for_outbound_sockets=127.24.227.193
--tserver_master_addrs=127.24.227.254:41135
--webserver_port=40821
--webserver_interface=127.24.227.193
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.193.keytab
--principal=kudu/127.24.227.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:33523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 23:16:51.041175 7837 log.cc:824] T 00000000000000000000000000000000 P 1c752c42c9f74c2491720c7a2b81880a: Log is configured to *not* fsync() on all Append() calls
I20260923 23:16:51.043691 7837 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c752c42c9f74c2491720c7a2b81880a: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 23:16:51.044035 7837 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c752c42c9f74c2491720c7a2b81880a: Bootstrap complete.
I20260923 23:16:51.047729 7837 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1c752c42c9f74c2491720c7a2b81880a [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c752c42c9f74c2491720c7a2b81880a" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41135 } }
I20260923 23:16:51.048137 7837 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1c752c42c9f74c2491720c7a2b81880a [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c752c42c9f74c2491720c7a2b81880a, State: Initialized, Role: FOLLOWER
I20260923 23:16:51.048319 7837 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1c752c42c9f74c2491720c7a2b81880a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c752c42c9f74c2491720c7a2b81880a" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41135 } }
I20260923 23:16:51.048400 7837 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1c752c42c9f74c2491720c7a2b81880a [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 23:16:51.048452 7837 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1c752c42c9f74c2491720c7a2b81880a [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 23:16:51.048506 7837 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1c752c42c9f74c2491720c7a2b81880a [term 2 FOLLOWER]: Advancing to term 3
I20260923 23:16:51.050088 7837 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1c752c42c9f74c2491720c7a2b81880a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c752c42c9f74c2491720c7a2b81880a" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41135 } }
I20260923 23:16:51.050220 7837 leader_election.cc:302] T 00000000000000000000000000000000 P 1c752c42c9f74c2491720c7a2b81880a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1c752c42c9f74c2491720c7a2b81880a; no voters:
I20260923 23:16:51.050400 7837 leader_election.cc:288] T 00000000000000000000000000000000 P 1c752c42c9f74c2491720c7a2b81880a [CANDIDATE]: Term 3 election: Requested vote from peers
I20260923 23:16:51.050428 7841 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1c752c42c9f74c2491720c7a2b81880a [term 3 FOLLOWER]: Leader election won for term 3
I20260923 23:16:51.050577 7841 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1c752c42c9f74c2491720c7a2b81880a [term 3 LEADER]: Becoming Leader. State: Replica: 1c752c42c9f74c2491720c7a2b81880a, State: Running, Role: LEADER
I20260923 23:16:51.050668 7837 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1c752c42c9f74c2491720c7a2b81880a [sys.catalog]: configured and running, proceeding with master startup.
I20260923 23:16:51.050662 7841 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1c752c42c9f74c2491720c7a2b81880a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c752c42c9f74c2491720c7a2b81880a" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41135 } }
I20260923 23:16:51.051075 7841 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c752c42c9f74c2491720c7a2b81880a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1c752c42c9f74c2491720c7a2b81880a. Latest consensus state: current_term: 3 leader_uuid: "1c752c42c9f74c2491720c7a2b81880a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c752c42c9f74c2491720c7a2b81880a" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41135 } } }
I20260923 23:16:51.051075 7842 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c752c42c9f74c2491720c7a2b81880a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "1c752c42c9f74c2491720c7a2b81880a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c752c42c9f74c2491720c7a2b81880a" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41135 } } }
I20260923 23:16:51.051239 7841 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c752c42c9f74c2491720c7a2b81880a [sys.catalog]: This master's current role is: LEADER
I20260923 23:16:51.051249 7842 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c752c42c9f74c2491720c7a2b81880a [sys.catalog]: This master's current role is: LEADER
I20260923 23:16:51.051575 7849 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 23:16:51.052441 7849 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 23:16:51.052716 7849 catalog_manager.cc:1259] Loaded cluster ID: 49a5d989716d404a8f46fc88738ceaf5
I20260923 23:16:51.052836 7849 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260923 23:16:51.053184 7849 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fc8a2a8a9ad google::LogMessage::Fail() at ??:0
@ 0x7fc8a2a8d88a google::LogMessage::SendToLog() at ??:0
@ 0x7fc8a2a8a4de google::LogMessage::Flush() at ??:0
@ 0x7fc8a2a8ba19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fc8aa9ab0f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7fc8aa9b48b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fc8a49580d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fc8a494d954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fc8a57b16db start_thread at ??:0
@ 0x7fc8a158861f clone at ??:0
W20260923 23:16:51.126493 7840 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:16:51.126704 7840 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:16:51.126752 7840 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:16:51.127856 7840 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:16:51.127956 7840 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:16:51.128003 7840 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.227.193
I20260923 23:16:51.129383 7840 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:33523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.193.keytab
--principal=kudu/127.24.227.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.227.193:44099
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.227.193
--webserver_port=40821
--tserver_master_addrs=127.24.227.254:41135
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7840
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.227.193
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:16:51.129633 7840 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:16:51.129848 7840 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:16:51.134502 7874 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:16:51.134593 7873 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:16:51.134490 7876 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:16:51.134835 7840 server_base.cc:1034] running on GCE node
I20260923 23:16:51.135057 7840 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:16:51.135304 7840 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:16:51.136520 7840 hybrid_clock.cc:648] HybridClock initialized: now 1790205411136497 us; error 30 us; skew 500 ppm
Sep 23 23:16:51 dist-test-slave-754n krb5kdc[6700](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205411, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:16:51.139338 7840 init.cc:377] Logged in from keytab as kudu/127.24.227.193@KRBTEST.COM (short username kudu)
I20260923 23:16:51.139827 7840 webserver.cc:460] Webserver started at http://127.24.227.193:40821/ using document root <none> and password file <none>
I20260923 23:16:51.140093 7840 fs_manager.cc:359] Metadata directory not provided
I20260923 23:16:51.140157 7840 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:16:51.141690 7840 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260923 23:16:51.142421 7883 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:16:51.142589 7840 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 23:16:51.142668 7840 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-0/wal
uuid: "06385beadc504000bfdbe0c62c0f09c2"
format_stamp: "Formatted at 2026-09-23 23:16:49 on dist-test-slave-754n"
server_key: "888e34ce2a70e59d063c8bbc4b1e42ae"
server_key_iv: "1be0c765f92a697e6dc06472fb31a01b"
server_key_version: "encryptionkey@0"
I20260923 23:16:51.142936 7840 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:16:51.171352 7840 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:16:51.173584 7840 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:16:51.173732 7840 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:16:51.173966 7840 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 23:16:51.174437 7890 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260923 23:16:51.175803 7840 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260923 23:16:51.175861 7840 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260923 23:16:51.175920 7840 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260923 23:16:51.176755 7840 ts_tablet_manager.cc:610] Registered 1 tablets
I20260923 23:16:51.176795 7840 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260923 23:16:51.176847 7890 tablet_bootstrap.cc:492] T 50ea81924f5242d3808257c752f058cb P 06385beadc504000bfdbe0c62c0f09c2: Bootstrap starting.
I20260923 23:16:51.184031 7840 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.193:44099
I20260923 23:16:51.184437 7840 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-0/data/info.pb
I20260923 23:16:51.185820 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 7840
I20260923 23:16:51.186081 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.227.194:34463
--local_ip_for_outbound_sockets=127.24.227.194
--tserver_master_addrs=127.24.227.254:41135
--webserver_port=40821
--webserver_interface=127.24.227.194
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.194.keytab
--principal=kudu/127.24.227.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:33523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 23:16:51.186288 7890 log.cc:824] T 50ea81924f5242d3808257c752f058cb P 06385beadc504000bfdbe0c62c0f09c2: Log is configured to *not* fsync() on all Append() calls
I20260923 23:16:51.186873 7997 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.193:44099 every 8 connection(s)
I20260923 23:16:51.188901 7890 tablet_bootstrap.cc:492] T 50ea81924f5242d3808257c752f058cb P 06385beadc504000bfdbe0c62c0f09c2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 23:16:51.189185 7890 tablet_bootstrap.cc:492] T 50ea81924f5242d3808257c752f058cb P 06385beadc504000bfdbe0c62c0f09c2: Bootstrap complete.
I20260923 23:16:51.190260 7890 ts_tablet_manager.cc:1397] T 50ea81924f5242d3808257c752f058cb P 06385beadc504000bfdbe0c62c0f09c2: Time spent bootstrapping tablet: real 0.013s user 0.001s sys 0.010s
I20260923 23:16:51.191833 7890 raft_consensus.cc:348] T 50ea81924f5242d3808257c752f058cb P 06385beadc504000bfdbe0c62c0f09c2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbd9c5173d840aca823fc791e513088" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 36069 } } peers { permanent_uuid: "06385beadc504000bfdbe0c62c0f09c2" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 44099 } } peers { permanent_uuid: "19e417d8de4a47d6b511eefbbe7043cf" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 34463 } }
I20260923 23:16:51.192271 7890 raft_consensus.cc:729] T 50ea81924f5242d3808257c752f058cb P 06385beadc504000bfdbe0c62c0f09c2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06385beadc504000bfdbe0c62c0f09c2, State: Initialized, Role: FOLLOWER
I20260923 23:16:51.192411 7890 consensus_queue.cc:260] T 50ea81924f5242d3808257c752f058cb P 06385beadc504000bfdbe0c62c0f09c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbd9c5173d840aca823fc791e513088" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 36069 } } peers { permanent_uuid: "06385beadc504000bfdbe0c62c0f09c2" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 44099 } } peers { permanent_uuid: "19e417d8de4a47d6b511eefbbe7043cf" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 34463 } }
I20260923 23:16:51.192700 7890 ts_tablet_manager.cc:1428] T 50ea81924f5242d3808257c752f058cb P 06385beadc504000bfdbe0c62c0f09c2: Time spent starting tablet: real 0.002s user 0.005s sys 0.000s
W20260923 23:16:51.256695 8000 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 23:16:51.185200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41135 (local address 127.24.227.193:51923)
0923 23:16:51.185402 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0923 23:16:51.185462 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0923 23:16:51.186447 (+ 985us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:16:51.256520 (+ 70073us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:41135: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":96,"threads_started":1}
W20260923 23:16:51.256956 7998 heartbeater.cc:644] Failed to heartbeat to 127.24.227.254:41135 (0 consecutive failures): Network error: Failed to ping master at 127.24.227.254:41135: Client connection negotiation failed: client connection to 127.24.227.254:41135: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260923 23:16:51.257277 8000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:51.257170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41135 (local address 127.24.227.193:41459)
0923 23:16:51.257218 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0923 23:16:51.257256 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:41135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260923 23:16:51.257643 8000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:51.257568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41135 (local address 127.24.227.193:44571)
0923 23:16:51.257609 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0923 23:16:51.257630 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:41135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260923 23:16:51.257759 7998 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260923 23:16:51.267555 8002 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:16:51.267745 8002 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:16:51.267779 8002 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:16:51.268894 8002 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:16:51.268970 8002 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:16:51.269013 8002 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.227.194
I20260923 23:16:51.270386 8002 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:33523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.194.keytab
--principal=kudu/127.24.227.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.227.194:34463
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.227.194
--webserver_port=40821
--tserver_master_addrs=127.24.227.254:41135
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8002
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.227.194
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:16:51.270623 8002 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:16:51.270830 8002 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:16:51.275313 8008 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:16:51.275305 8011 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:16:51.275290 8009 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:16:51.275440 8002 server_base.cc:1034] running on GCE node
I20260923 23:16:51.275703 8002 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:16:51.275982 8002 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:16:51.277130 8002 hybrid_clock.cc:648] HybridClock initialized: now 1790205411277121 us; error 35 us; skew 500 ppm
Sep 23 23:16:51 dist-test-slave-754n krb5kdc[6700](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205411, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:16:51.279481 8002 init.cc:377] Logged in from keytab as kudu/127.24.227.194@KRBTEST.COM (short username kudu)
I20260923 23:16:51.279945 8002 webserver.cc:460] Webserver started at http://127.24.227.194:40821/ using document root <none> and password file <none>
I20260923 23:16:51.280165 8002 fs_manager.cc:359] Metadata directory not provided
I20260923 23:16:51.280227 8002 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:16:51.281373 8002 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 23:16:51.281883 8018 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:16:51.282042 8002 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 23:16:51.282130 8002 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-1/wal
uuid: "19e417d8de4a47d6b511eefbbe7043cf"
format_stamp: "Formatted at 2026-09-23 23:16:49 on dist-test-slave-754n"
server_key: "18f0ee9a2d01d9996329afa39a60c510"
server_key_iv: "f124b9ef428a1cb011648191998b03ab"
server_key_version: "encryptionkey@0"
I20260923 23:16:51.282444 8002 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:16:51.314090 8002 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:16:51.316200 8002 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:16:51.316344 8002 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:16:51.316557 8002 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 23:16:51.316826 8002 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 23:16:51.316872 8002 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 23:16:51.316911 8002 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 23:16:51.316939 8002 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 23:16:51.322294 8002 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.194:34463
I20260923 23:16:51.322327 8131 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.194:34463 every 8 connection(s)
I20260923 23:16:51.322660 8002 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-1/data/info.pb
I20260923 23:16:51.323580 8134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:51.323257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41135 (local address 127.24.227.194:42575)
0923 23:16:51.323385 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0923 23:16:51.323459 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:41135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":67,"threads_started":1}
W20260923 23:16:51.323944 8132 heartbeater.cc:644] Failed to heartbeat to 127.24.227.254:41135 (0 consecutive failures): Network error: Failed to ping master at 127.24.227.254:41135: Client connection negotiation failed: client connection to 127.24.227.254:41135: connect: Connection refused (error 111)
I20260923 23:16:51.324196 8134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:51.324116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41135 (local address 127.24.227.194:59935)
0923 23:16:51.324155 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0923 23:16:51.324181 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:41135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260923 23:16:51.324447 8134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:51.324392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41135 (local address 127.24.227.194:58477)
0923 23:16:51.324419 (+ 27us) negotiation.cc:107] Waiting for socket to connect
0923 23:16:51.324436 (+ 17us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:41135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":11}
W20260923 23:16:51.324566 8132 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260923 23:16:51.331615 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 8002
I20260923 23:16:51.331892 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.227.195:36069
--local_ip_for_outbound_sockets=127.24.227.195
--tserver_master_addrs=127.24.227.254:41135
--webserver_port=36787
--webserver_interface=127.24.227.195
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.195.keytab
--principal=kudu/127.24.227.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:33523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:16:51.410225 8135 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:16:51.410418 8135 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:16:51.410447 8135 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:16:51.411510 8135 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:16:51.411557 8135 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:16:51.411582 8135 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.227.195
I20260923 23:16:51.413017 8135 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:33523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.195.keytab
--principal=kudu/127.24.227.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.227.195:36069
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.227.195
--webserver_port=36787
--tserver_master_addrs=127.24.227.254:41135
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8135
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.227.195
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:16:51.413211 8135 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:16:51.413353 8135 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:16:51.417894 8140 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:16:51.417954 8141 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:16:51.417896 8143 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:16:51.418085 8135 server_base.cc:1034] running on GCE node
I20260923 23:16:51.418285 8135 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:16:51.418484 8135 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:16:51.419626 8135 hybrid_clock.cc:648] HybridClock initialized: now 1790205411419617 us; error 26 us; skew 500 ppm
Sep 23 23:16:51 dist-test-slave-754n krb5kdc[6700](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205411, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:16:51.421975 8135 init.cc:377] Logged in from keytab as kudu/127.24.227.195@KRBTEST.COM (short username kudu)
I20260923 23:16:51.422405 8135 webserver.cc:460] Webserver started at http://127.24.227.195:36787/ using document root <none> and password file <none>
I20260923 23:16:51.422628 8135 fs_manager.cc:359] Metadata directory not provided
I20260923 23:16:51.422672 8135 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:16:51.423920 8135 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 23:16:51.424454 8150 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:16:51.424619 8135 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 23:16:51.424697 8135 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-2/wal
uuid: "1dbd9c5173d840aca823fc791e513088"
format_stamp: "Formatted at 2026-09-23 23:16:49 on dist-test-slave-754n"
server_key: "9dfabea4b8a7cc86f59b5eeac6e2275e"
server_key_iv: "ed1e42305954f38553aae7d94e7497b9"
server_key_version: "encryptionkey@0"
I20260923 23:16:51.424973 8135 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:16:51.437549 8135 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:16:51.438998 8135 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:16:51.439126 8135 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:16:51.439291 8135 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 23:16:51.439558 8135 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 23:16:51.439587 8135 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 23:16:51.439621 8135 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 23:16:51.439646 8135 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 23:16:51.444989 8135 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.195:36069
I20260923 23:16:51.445032 8263 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.195:36069 every 8 connection(s)
I20260923 23:16:51.445315 8135 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790205259943518-25487-0/minicluster-data/ts-2/data/info.pb
I20260923 23:16:51.446213 8266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:51.445863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41135 (local address 127.24.227.195:45897)
0923 23:16:51.445983 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0923 23:16:51.446050 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:41135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":98,"threads_started":1}
I20260923 23:16:51.446281 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 8135
W20260923 23:16:51.446617 8264 heartbeater.cc:644] Failed to heartbeat to 127.24.227.254:41135 (0 consecutive failures): Network error: Failed to ping master at 127.24.227.254:41135: Client connection negotiation failed: client connection to 127.24.227.254:41135: connect: Connection refused (error 111)
I20260923 23:16:51.446903 8266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:51.446802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41135 (local address 127.24.227.195:50945)
0923 23:16:51.446862 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0923 23:16:51.446887 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:41135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260923 23:16:51.447152 8266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:51.447105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41135 (local address 127.24.227.195:40273)
0923 23:16:51.447123 (+ 18us) negotiation.cc:107] Waiting for socket to connect
0923 23:16:51.447140 (+ 17us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:41135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":10}
W20260923 23:16:51.447247 8264 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260923 23:16:52.258301 8268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:52.258056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41135 (local address 127.24.227.193:37089)
0923 23:16:52.258199 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0923 23:16:52.258233 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:41135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":61,"threads_started":1}
I20260923 23:16:52.324955 8269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:52.324796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41135 (local address 127.24.227.194:59811)
0923 23:16:52.324885 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0923 23:16:52.324911 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:41135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77,"thread_start_us":48,"threads_started":1}
I20260923 23:16:52.447726 8270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:52.447536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41135 (local address 127.24.227.195:37521)
0923 23:16:52.447646 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0923 23:16:52.447677 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:41135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":52,"threads_started":1}
I20260923 23:16:52.832113 8271 raft_consensus.cc:482] T 50ea81924f5242d3808257c752f058cb P 06385beadc504000bfdbe0c62c0f09c2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 23:16:52.832223 8271 raft_consensus.cc:504] T 50ea81924f5242d3808257c752f058cb P 06385beadc504000bfdbe0c62c0f09c2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbd9c5173d840aca823fc791e513088" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 36069 } } peers { permanent_uuid: "06385beadc504000bfdbe0c62c0f09c2" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 44099 } } peers { permanent_uuid: "19e417d8de4a47d6b511eefbbe7043cf" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 34463 } }
I20260923 23:16:52.832556 8271 leader_election.cc:288] T 50ea81924f5242d3808257c752f058cb P 06385beadc504000bfdbe0c62c0f09c2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1dbd9c5173d840aca823fc791e513088 (127.24.227.195:36069), 19e417d8de4a47d6b511eefbbe7043cf (127.24.227.194:34463)
Sep 23 23:16:52 dist-test-slave-754n krb5kdc[6700](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205411, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.193@KRBTEST.COM for kudu/127.24.227.194@KRBTEST.COM
Sep 23 23:16:52 dist-test-slave-754n krb5kdc[6700](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205411, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.193@KRBTEST.COM for kudu/127.24.227.195@KRBTEST.COM
I20260923 23:16:52.842710 8272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:52.832704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.195:36069 (local address 127.24.227.193:50989)
0923 23:16:52.832814 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0923 23:16:52.832825 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 23:16:52.832882 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:16:52.833724 (+ 842us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:16:52.833727 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:16:52.834127 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:16:52.834305 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:16:52.834311 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:16:52.835549 (+ 1238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:16:52.835551 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:16:52.836199 (+ 648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:16:52.836202 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:16:52.836775 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:16:52.836776 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:16:52.836881 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:16:52.837485 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:16:52.837495 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:16:52.839366 (+ 1871us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:16:52.841261 (+ 1895us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:16:52.841261 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:16:52.841262 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:16:52.841499 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:16:52.841725 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:16:52.841726 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:16:52.841726 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:16:52.841812 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:16:52.842218 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:16:52.842224 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:16:52.842340 (+ 116us) client_negotiation.cc:769] Sending connection context
0923 23:16:52.842445 (+ 105us) client_negotiation.cc:240] Negotiation successful
0923 23:16:52.842614 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95,"mutex_wait_us":40,"thread_start_us":71,"threads_started":1}
I20260923 23:16:52.842723 8273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:52.832818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.193:50989 (local address 127.24.227.195:36069)
0923 23:16:52.833026 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0923 23:16:52.833036 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 23:16:52.833065 (+ 29us) server_negotiation.cc:407] Connection header received
0923 23:16:52.833514 (+ 449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:16:52.833516 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:16:52.833557 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:16:52.833699 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:16:52.834428 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:52.835435 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:52.836282 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:52.836684 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:52.836814 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:16:52.839457 (+ 2643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:16:52.839460 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:16:52.839460 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:16:52.839468 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:16:52.841167 (+ 1699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:16:52.841574 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:16:52.841576 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:16:52.841577 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:16:52.841657 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:16:52.841875 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:16:52.841876 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:16:52.841877 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:16:52.842149 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:16:52.842235 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0923 23:16:52.842505 (+ 270us) server_negotiation.cc:299] Negotiation successful
0923 23:16:52.842621 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":71,"threads_started":1}
I20260923 23:16:52.842731 8275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:52.832972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.193:42529 (local address 127.24.227.194:34463)
0923 23:16:52.833116 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0923 23:16:52.833127 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0923 23:16:52.833159 (+ 32us) server_negotiation.cc:407] Connection header received
0923 23:16:52.833560 (+ 401us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:16:52.833562 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:16:52.833597 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:16:52.833760 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:16:52.834424 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:52.835279 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:52.836293 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:16:52.836682 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:16:52.836798 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:16:52.839062 (+ 2264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:16:52.839065 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:16:52.839065 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:16:52.839073 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:16:52.841087 (+ 2014us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:16:52.841543 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:16:52.841546 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:16:52.841546 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:16:52.841642 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:16:52.841872 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:16:52.841873 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:16:52.841873 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:16:52.842149 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:16:52.842235 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0923 23:16:52.842486 (+ 251us) server_negotiation.cc:299] Negotiation successful
0923 23:16:52.842608 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":50,"threads_started":1}
I20260923 23:16:52.842813 8274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:52.832879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.194:34463 (local address 127.24.227.193:42529)
0923 23:16:52.833016 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0923 23:16:52.833027 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 23:16:52.833088 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:16:52.833735 (+ 647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:16:52.833736 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:16:52.834126 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:16:52.834305 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:16:52.834311 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:16:52.835374 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:16:52.835374 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:16:52.836198 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:16:52.836200 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:16:52.836769 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:16:52.836770 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:16:52.836882 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:16:52.837433 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:16:52.837441 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:16:52.838974 (+ 1533us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:16:52.841189 (+ 2215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:16:52.841196 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:16:52.841198 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:16:52.841460 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:16:52.841706 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:16:52.841707 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:16:52.841707 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:16:52.841810 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:16:52.842216 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:16:52.842222 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:16:52.842341 (+ 119us) client_negotiation.cc:769] Sending connection context
0923 23:16:52.842437 (+ 96us) client_negotiation.cc:240] Negotiation successful
0923 23:16:52.842688 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"mutex_wait_us":102,"thread_start_us":84,"threads_started":1}
W20260923 23:16:52.844005 7887 leader_election.cc:341] T 50ea81924f5242d3808257c752f058cb P 06385beadc504000bfdbe0c62c0f09c2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 19e417d8de4a47d6b511eefbbe7043cf (127.24.227.194:34463): Not found: Tablet not found: 50ea81924f5242d3808257c752f058cb
W20260923 23:16:52.844069 7887 leader_election.cc:341] T 50ea81924f5242d3808257c752f058cb P 06385beadc504000bfdbe0c62c0f09c2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1dbd9c5173d840aca823fc791e513088 (127.24.227.195:36069): Not found: Tablet not found: 50ea81924f5242d3808257c752f058cb
I20260923 23:16:52.844085 7887 leader_election.cc:302] T 50ea81924f5242d3808257c752f058cb P 06385beadc504000bfdbe0c62c0f09c2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06385beadc504000bfdbe0c62c0f09c2; no voters: 19e417d8de4a47d6b511eefbbe7043cf, 1dbd9c5173d840aca823fc791e513088
I20260923 23:16:52.844168 8271 raft_consensus.cc:2726] T 50ea81924f5242d3808257c752f058cb P 06385beadc504000bfdbe0c62c0f09c2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260923 23:16:53.258810 8274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:53.258698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41135 (local address 127.24.227.193:53099)
0923 23:16:53.258745 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0923 23:16:53.258775 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:41135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260923 23:16:53.325482 8276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:53.325302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41135 (local address 127.24.227.194:33367)
0923 23:16:53.325407 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0923 23:16:53.325444 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:41135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":51,"threads_started":1}
I20260923 23:16:53.448397 8277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:53.448210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41135 (local address 127.24.227.195:44241)
0923 23:16:53.448331 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0923 23:16:53.448364 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:41135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":67,"threads_started":1}
I20260923 23:16:54.259553 8278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:54.259314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41135 (local address 127.24.227.193:50439)
0923 23:16:54.259443 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0923 23:16:54.259479 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:41135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":56,"threads_started":1}
I20260923 23:16:54.326120 8279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:54.325943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41135 (local address 127.24.227.194:39717)
0923 23:16:54.326052 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0923 23:16:54.326084 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:41135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":56,"threads_started":1}
I20260923 23:16:54.448912 8280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:54.448750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41135 (local address 127.24.227.195:34065)
0923 23:16:54.448851 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0923 23:16:54.448881 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:41135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86,"thread_start_us":53,"threads_started":1}
I20260923 23:16:54.610639 8281 raft_consensus.cc:482] T 50ea81924f5242d3808257c752f058cb P 06385beadc504000bfdbe0c62c0f09c2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 23:16:54.610725 8281 raft_consensus.cc:504] T 50ea81924f5242d3808257c752f058cb P 06385beadc504000bfdbe0c62c0f09c2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbd9c5173d840aca823fc791e513088" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 36069 } } peers { permanent_uuid: "06385beadc504000bfdbe0c62c0f09c2" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 44099 } } peers { permanent_uuid: "19e417d8de4a47d6b511eefbbe7043cf" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 34463 } }
I20260923 23:16:54.610836 8281 leader_election.cc:288] T 50ea81924f5242d3808257c752f058cb P 06385beadc504000bfdbe0c62c0f09c2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1dbd9c5173d840aca823fc791e513088 (127.24.227.195:36069), 19e417d8de4a47d6b511eefbbe7043cf (127.24.227.194:34463)
W20260923 23:16:54.611312 7887 leader_election.cc:341] T 50ea81924f5242d3808257c752f058cb P 06385beadc504000bfdbe0c62c0f09c2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1dbd9c5173d840aca823fc791e513088 (127.24.227.195:36069): Not found: Tablet not found: 50ea81924f5242d3808257c752f058cb
W20260923 23:16:54.611375 7887 leader_election.cc:341] T 50ea81924f5242d3808257c752f058cb P 06385beadc504000bfdbe0c62c0f09c2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 19e417d8de4a47d6b511eefbbe7043cf (127.24.227.194:34463): Not found: Tablet not found: 50ea81924f5242d3808257c752f058cb
I20260923 23:16:54.611459 7887 leader_election.cc:302] T 50ea81924f5242d3808257c752f058cb P 06385beadc504000bfdbe0c62c0f09c2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06385beadc504000bfdbe0c62c0f09c2; no voters: 19e417d8de4a47d6b511eefbbe7043cf, 1dbd9c5173d840aca823fc791e513088
I20260923 23:16:54.611558 8281 raft_consensus.cc:2726] T 50ea81924f5242d3808257c752f058cb P 06385beadc504000bfdbe0c62c0f09c2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260923 23:16:55.260214 8282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:55.260009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41135 (local address 127.24.227.193:48341)
0923 23:16:55.260141 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0923 23:16:55.260174 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:41135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":74,"threads_started":1}
I20260923 23:16:55.326723 8283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:55.326540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41135 (local address 127.24.227.194:51817)
0923 23:16:55.326643 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0923 23:16:55.326675 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:41135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":53,"threads_started":1}
I20260923 23:16:55.449499 8284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:55.449318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41135 (local address 127.24.227.195:45757)
0923 23:16:55.449431 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0923 23:16:55.449458 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:41135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":61,"threads_started":1}
I20260923 23:16:56.126466 8285 raft_consensus.cc:482] T 50ea81924f5242d3808257c752f058cb P 06385beadc504000bfdbe0c62c0f09c2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 23:16:56.126559 8285 raft_consensus.cc:504] T 50ea81924f5242d3808257c752f058cb P 06385beadc504000bfdbe0c62c0f09c2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbd9c5173d840aca823fc791e513088" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 36069 } } peers { permanent_uuid: "06385beadc504000bfdbe0c62c0f09c2" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 44099 } } peers { permanent_uuid: "19e417d8de4a47d6b511eefbbe7043cf" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 34463 } }
I20260923 23:16:56.126747 8285 leader_election.cc:288] T 50ea81924f5242d3808257c752f058cb P 06385beadc504000bfdbe0c62c0f09c2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1dbd9c5173d840aca823fc791e513088 (127.24.227.195:36069), 19e417d8de4a47d6b511eefbbe7043cf (127.24.227.194:34463)
W20260923 23:16:56.127123 7887 leader_election.cc:341] T 50ea81924f5242d3808257c752f058cb P 06385beadc504000bfdbe0c62c0f09c2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 19e417d8de4a47d6b511eefbbe7043cf (127.24.227.194:34463): Not found: Tablet not found: 50ea81924f5242d3808257c752f058cb
W20260923 23:16:56.127190 7887 leader_election.cc:341] T 50ea81924f5242d3808257c752f058cb P 06385beadc504000bfdbe0c62c0f09c2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1dbd9c5173d840aca823fc791e513088 (127.24.227.195:36069): Not found: Tablet not found: 50ea81924f5242d3808257c752f058cb
I20260923 23:16:56.127205 7887 leader_election.cc:302] T 50ea81924f5242d3808257c752f058cb P 06385beadc504000bfdbe0c62c0f09c2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06385beadc504000bfdbe0c62c0f09c2; no voters: 19e417d8de4a47d6b511eefbbe7043cf, 1dbd9c5173d840aca823fc791e513088
I20260923 23:16:56.127291 8285 raft_consensus.cc:2726] T 50ea81924f5242d3808257c752f058cb P 06385beadc504000bfdbe0c62c0f09c2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260923 23:16:56.260892 8286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:56.260605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41135 (local address 127.24.227.193:43679)
0923 23:16:56.260816 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0923 23:16:56.260850 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:41135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":67,"threads_started":1}
I20260923 23:16:56.327337 8287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:56.327154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41135 (local address 127.24.227.194:57287)
0923 23:16:56.327259 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0923 23:16:56.327302 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:41135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":57,"threads_started":1}
I20260923 23:16:56.450064 8288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:56.449885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41135 (local address 127.24.227.195:48531)
0923 23:16:56.449998 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0923 23:16:56.450029 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:41135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":55,"threads_started":1}
I20260923 23:16:57.261495 8289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:57.261309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41135 (local address 127.24.227.193:47863)
0923 23:16:57.261421 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0923 23:16:57.261449 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:41135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":57,"threads_started":1}
I20260923 23:16:57.327898 8290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:57.327713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41135 (local address 127.24.227.194:55907)
0923 23:16:57.327817 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0923 23:16:57.327847 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:41135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":51,"threads_started":1}
I20260923 23:16:57.450635 8291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:57.450434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41135 (local address 127.24.227.195:52311)
0923 23:16:57.450567 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0923 23:16:57.450599 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:41135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":77,"threads_started":1}
I20260923 23:16:58.262070 8292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:58.261872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41135 (local address 127.24.227.193:60459)
0923 23:16:58.261995 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0923 23:16:58.262028 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:41135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":55,"threads_started":1}
I20260923 23:16:58.328544 8293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:58.328317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41135 (local address 127.24.227.194:50077)
0923 23:16:58.328475 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0923 23:16:58.328507 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:41135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":68,"threads_started":1}
I20260923 23:16:58.451290 8294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:58.451049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41135 (local address 127.24.227.195:42569)
0923 23:16:58.451191 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0923 23:16:58.451229 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:41135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85,"thread_start_us":59,"threads_started":1}
I20260923 23:16:58.540343 8295 raft_consensus.cc:482] T 50ea81924f5242d3808257c752f058cb P 06385beadc504000bfdbe0c62c0f09c2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 23:16:58.540436 8295 raft_consensus.cc:504] T 50ea81924f5242d3808257c752f058cb P 06385beadc504000bfdbe0c62c0f09c2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbd9c5173d840aca823fc791e513088" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 36069 } } peers { permanent_uuid: "06385beadc504000bfdbe0c62c0f09c2" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 44099 } } peers { permanent_uuid: "19e417d8de4a47d6b511eefbbe7043cf" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 34463 } }
I20260923 23:16:58.540640 8295 leader_election.cc:288] T 50ea81924f5242d3808257c752f058cb P 06385beadc504000bfdbe0c62c0f09c2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1dbd9c5173d840aca823fc791e513088 (127.24.227.195:36069), 19e417d8de4a47d6b511eefbbe7043cf (127.24.227.194:34463)
W20260923 23:16:58.540992 7887 leader_election.cc:341] T 50ea81924f5242d3808257c752f058cb P 06385beadc504000bfdbe0c62c0f09c2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1dbd9c5173d840aca823fc791e513088 (127.24.227.195:36069): Not found: Tablet not found: 50ea81924f5242d3808257c752f058cb
W20260923 23:16:58.541061 7887 leader_election.cc:341] T 50ea81924f5242d3808257c752f058cb P 06385beadc504000bfdbe0c62c0f09c2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 19e417d8de4a47d6b511eefbbe7043cf (127.24.227.194:34463): Not found: Tablet not found: 50ea81924f5242d3808257c752f058cb
I20260923 23:16:58.541105 7887 leader_election.cc:302] T 50ea81924f5242d3808257c752f058cb P 06385beadc504000bfdbe0c62c0f09c2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06385beadc504000bfdbe0c62c0f09c2; no voters: 19e417d8de4a47d6b511eefbbe7043cf, 1dbd9c5173d840aca823fc791e513088
I20260923 23:16:58.541208 8295 raft_consensus.cc:2726] T 50ea81924f5242d3808257c752f058cb P 06385beadc504000bfdbe0c62c0f09c2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260923 23:16:59.262665 8296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:59.262477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41135 (local address 127.24.227.193:54825)
0923 23:16:59.262585 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0923 23:16:59.262620 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:41135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":54,"threads_started":1}
I20260923 23:16:59.329101 8297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:59.328922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41135 (local address 127.24.227.194:41865)
0923 23:16:59.329038 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0923 23:16:59.329068 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:41135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":56,"threads_started":1}
I20260923 23:16:59.451848 8298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:16:59.451671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41135 (local address 127.24.227.195:57513)
0923 23:16:59.451781 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0923 23:16:59.451812 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:41135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":56,"threads_started":1}
I20260923 23:17:00.263340 8299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:17:00.263108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41135 (local address 127.24.227.193:49157)
0923 23:17:00.263255 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0923 23:17:00.263296 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:41135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":84,"threads_started":1}
I20260923 23:17:00.329708 8300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:17:00.329486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41135 (local address 127.24.227.194:54493)
0923 23:17:00.329637 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0923 23:17:00.329669 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:41135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":58,"threads_started":1}
I20260923 23:17:00.339735 8301 raft_consensus.cc:482] T 50ea81924f5242d3808257c752f058cb P 06385beadc504000bfdbe0c62c0f09c2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 23:17:00.339808 8301 raft_consensus.cc:504] T 50ea81924f5242d3808257c752f058cb P 06385beadc504000bfdbe0c62c0f09c2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbd9c5173d840aca823fc791e513088" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 36069 } } peers { permanent_uuid: "06385beadc504000bfdbe0c62c0f09c2" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 44099 } } peers { permanent_uuid: "19e417d8de4a47d6b511eefbbe7043cf" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 34463 } }
I20260923 23:17:00.340027 8301 leader_election.cc:288] T 50ea81924f5242d3808257c752f058cb P 06385beadc504000bfdbe0c62c0f09c2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1dbd9c5173d840aca823fc791e513088 (127.24.227.195:36069), 19e417d8de4a47d6b511eefbbe7043cf (127.24.227.194:34463)
W20260923 23:17:00.340354 7887 leader_election.cc:341] T 50ea81924f5242d3808257c752f058cb P 06385beadc504000bfdbe0c62c0f09c2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 19e417d8de4a47d6b511eefbbe7043cf (127.24.227.194:34463): Not found: Tablet not found: 50ea81924f5242d3808257c752f058cb
W20260923 23:17:00.340456 7887 leader_election.cc:341] T 50ea81924f5242d3808257c752f058cb P 06385beadc504000bfdbe0c62c0f09c2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1dbd9c5173d840aca823fc791e513088 (127.24.227.195:36069): Not found: Tablet not found: 50ea81924f5242d3808257c752f058cb
I20260923 23:17:00.340489 7887 leader_election.cc:302] T 50ea81924f5242d3808257c752f058cb P 06385beadc504000bfdbe0c62c0f09c2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06385beadc504000bfdbe0c62c0f09c2; no voters: 19e417d8de4a47d6b511eefbbe7043cf, 1dbd9c5173d840aca823fc791e513088
I20260923 23:17:00.340574 8301 raft_consensus.cc:2726] T 50ea81924f5242d3808257c752f058cb P 06385beadc504000bfdbe0c62c0f09c2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260923 23:17:00.452474 8302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:17:00.452257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41135 (local address 127.24.227.195:35657)
0923 23:17:00.452404 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0923 23:17:00.452438 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:41135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":72,"threads_started":1}
I20260923 23:17:01.264092 8303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:17:01.263916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41135 (local address 127.24.227.193:53369)
0923 23:17:01.264021 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0923 23:17:01.264053 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:41135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":54,"threads_started":1}
I20260923 23:17:01.330260 8304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:17:01.330100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41135 (local address 127.24.227.194:40237)
0923 23:17:01.330197 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0923 23:17:01.330224 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:41135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86,"thread_start_us":54,"threads_started":1}
I20260923 23:17:01.453073 8305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:17:01.452882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41135 (local address 127.24.227.195:41225)
0923 23:17:01.452995 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0923 23:17:01.453037 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:41135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":63,"threads_started":1}
I20260923 23:17:02.162330 8306 raft_consensus.cc:482] T 50ea81924f5242d3808257c752f058cb P 06385beadc504000bfdbe0c62c0f09c2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 23:17:02.162415 8306 raft_consensus.cc:504] T 50ea81924f5242d3808257c752f058cb P 06385beadc504000bfdbe0c62c0f09c2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dbd9c5173d840aca823fc791e513088" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 36069 } } peers { permanent_uuid: "06385beadc504000bfdbe0c62c0f09c2" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 44099 } } peers { permanent_uuid: "19e417d8de4a47d6b511eefbbe7043cf" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 34463 } }
I20260923 23:17:02.162604 8306 leader_election.cc:288] T 50ea81924f5242d3808257c752f058cb P 06385beadc504000bfdbe0c62c0f09c2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1dbd9c5173d840aca823fc791e513088 (127.24.227.195:36069), 19e417d8de4a47d6b511eefbbe7043cf (127.24.227.194:34463)
W20260923 23:17:02.163008 7887 leader_election.cc:341] T 50ea81924f5242d3808257c752f058cb P 06385beadc504000bfdbe0c62c0f09c2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1dbd9c5173d840aca823fc791e513088 (127.24.227.195:36069): Not found: Tablet not found: 50ea81924f5242d3808257c752f058cb
W20260923 23:17:02.163059 7887 leader_election.cc:341] T 50ea81924f5242d3808257c752f058cb P 06385beadc504000bfdbe0c62c0f09c2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 19e417d8de4a47d6b511eefbbe7043cf (127.24.227.194:34463): Not found: Tablet not found: 50ea81924f5242d3808257c752f058cb
I20260923 23:17:02.163074 7887 leader_election.cc:302] T 50ea81924f5242d3808257c752f058cb P 06385beadc504000bfdbe0c62c0f09c2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06385beadc504000bfdbe0c62c0f09c2; no voters: 19e417d8de4a47d6b511eefbbe7043cf, 1dbd9c5173d840aca823fc791e513088
I20260923 23:17:02.163161 8306 raft_consensus.cc:2726] T 50ea81924f5242d3808257c752f058cb P 06385beadc504000bfdbe0c62c0f09c2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260923 23:17:02.264740 8307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:17:02.264546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41135 (local address 127.24.227.193:49903)
0923 23:17:02.264670 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0923 23:17:02.264700 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:41135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":59,"threads_started":1}
I20260923 23:17:02.330806 8308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:17:02.330619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41135 (local address 127.24.227.194:53701)
0923 23:17:02.330731 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0923 23:17:02.330771 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:41135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":57,"threads_started":1}
I20260923 23:17:02.453650 8309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:17:02.453465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41135 (local address 127.24.227.195:40265)
0923 23:17:02.453570 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0923 23:17:02.453598 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:41135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":53,"threads_started":1}
I20260923 23:17:03.265347 8310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:17:03.265135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41135 (local address 127.24.227.193:44897)
0923 23:17:03.265269 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0923 23:17:03.265304 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:41135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":55,"threads_started":1}
I20260923 23:17:03.331389 8311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:17:03.331178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41135 (local address 127.24.227.194:60519)
0923 23:17:03.331302 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0923 23:17:03.331349 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:41135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":68,"threads_started":1}
I20260923 23:17:03.454259 8312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:17:03.454062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41135 (local address 127.24.227.195:34211)
0923 23:17:03.454183 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0923 23:17:03.454213 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:41135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":68,"threads_started":1}
I20260923 23:17:04.266099 8313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:17:04.265894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41135 (local address 127.24.227.193:58345)
0923 23:17:04.266021 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0923 23:17:04.266055 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:41135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":58,"threads_started":1}
I20260923 23:17:04.332077 8314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:17:04.331839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41135 (local address 127.24.227.194:58477)
0923 23:17:04.332005 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0923 23:17:04.332039 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:41135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":102,"threads_started":1}
I20260923 23:17:04.454900 8315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:17:04.454715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41135 (local address 127.24.227.195:39201)
0923 23:17:04.454828 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0923 23:17:04.454860 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:41135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":59,"threads_started":1}
I20260923 23:17:05.266892 8316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:17:05.266678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41135 (local address 127.24.227.193:39899)
0923 23:17:05.266813 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0923 23:17:05.266848 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:41135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":54,"threads_started":1}
I20260923 23:17:05.332641 8317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:17:05.332469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41135 (local address 127.24.227.194:40711)
0923 23:17:05.332576 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0923 23:17:05.332606 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:41135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":54,"threads_started":1}
I20260923 23:17:05.455497 8318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:17:05.455325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41135 (local address 127.24.227.195:41985)
0923 23:17:05.455430 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0923 23:17:05.455459 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:41135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":56,"threads_started":1}
I20260923 23:17:06.267568 8319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:17:06.267371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41135 (local address 127.24.227.193:38855)
0923 23:17:06.267483 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0923 23:17:06.267513 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:41135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":55,"threads_started":1}
I20260923 23:17:06.333282 8320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:17:06.333070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41135 (local address 127.24.227.194:57581)
0923 23:17:06.333211 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0923 23:17:06.333245 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:41135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":68,"threads_started":1}
I20260923 23:17:06.452955 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 7840
I20260923 23:17:06.456154 8321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:17:06.455981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41135 (local address 127.24.227.195:55647)
0923 23:17:06.456088 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0923 23:17:06.456120 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:41135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87,"thread_start_us":64,"threads_started":1}
I20260923 23:17:06.457552 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 8002
I20260923 23:17:06.461527 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 8135
2026-09-23T23:17:06Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18297 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 23:17:06 dist-test-slave-754n krb5kdc[8324](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 23:17:06 dist-test-slave-754n krb5kdc[8324](info): set up 2 sockets
Sep 23 23:17:06 dist-test-slave-754n krb5kdc[8324](info): commencing operation
krb5kdc: starting...
W20260923 23:17:07.471851 25487 mini_kdc.cc:121] Time spent starting KDC: real 0.991s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/krb5kdc/test-user.keytab.
Sep 23 23:17:07 dist-test-slave-754n krb5kdc[8324](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205427, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T23:17:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T23:17:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.227.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.227.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.227.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab.
Entry for principal kudu/127.24.227.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab.
I20260923 23:17:07.586879 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.254:40121
--webserver_interface=127.24.227.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab
--principal=kudu/127.24.227.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:33557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.227.254:40121
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:17:07.662127 8337 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:17:07.662341 8337 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:17:07.662376 8337 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:17:07.663523 8337 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260923 23:17:07.663599 8337 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260923 23:17:07.663619 8337 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260923 23:17:07.663636 8337 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 23:17:07.663664 8337 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:17:07.663693 8337 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:17:07.663715 8337 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 23:17:07.663734 8337 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 23:17:07.665241 8337 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:33557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.24.227.254:40121
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab
--principal=kudu/127.24.227.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.254:40121
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.227.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8337
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:17:07.665529 8337 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:17:07.665771 8337 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:17:07.670663 8342 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:17:07.670688 8345 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:17:07.670710 8343 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:17:07.670770 8337 server_base.cc:1034] running on GCE node
I20260923 23:17:07.671201 8337 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:17:07.671481 8337 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:17:07.672653 8337 hybrid_clock.cc:648] HybridClock initialized: now 1790205427672632 us; error 37 us; skew 500 ppm
Sep 23 23:17:07 dist-test-slave-754n krb5kdc[8324](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205427, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:17:07.674907 8337 init.cc:377] Logged in from keytab as kudu/127.24.227.254@KRBTEST.COM (short username kudu)
I20260923 23:17:07.675345 8337 webserver.cc:460] Webserver started at http://127.24.227.254:41419/ using document root <none> and password file <none>
I20260923 23:17:07.675565 8337 fs_manager.cc:359] Metadata directory not provided
I20260923 23:17:07.675631 8337 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:17:07.675757 8337 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:17:07.676774 8337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/master-0/data/instance:
uuid: "70625401bbe84b8faa08091c510ffafe"
format_stamp: "Formatted at 2026-09-23 23:17:07 on dist-test-slave-754n"
server_key: "0315412c130e13af12293caf8c67a1a2"
server_key_iv: "61d15ddcff79eeabd9a47b62fdc13afd"
server_key_version: "encryptionkey@0"
I20260923 23:17:07.677131 8337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/master-0/wal/instance:
uuid: "70625401bbe84b8faa08091c510ffafe"
format_stamp: "Formatted at 2026-09-23 23:17:07 on dist-test-slave-754n"
server_key: "0315412c130e13af12293caf8c67a1a2"
server_key_iv: "61d15ddcff79eeabd9a47b62fdc13afd"
server_key_version: "encryptionkey@0"
I20260923 23:17:07.678392 8337 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260923 23:17:07.679087 8352 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:17:07.679255 8337 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 23:17:07.679345 8337 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/master-0/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/master-0/wal
uuid: "70625401bbe84b8faa08091c510ffafe"
format_stamp: "Formatted at 2026-09-23 23:17:07 on dist-test-slave-754n"
server_key: "0315412c130e13af12293caf8c67a1a2"
server_key_iv: "61d15ddcff79eeabd9a47b62fdc13afd"
server_key_version: "encryptionkey@0"
I20260923 23:17:07.679427 8337 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:17:07.693789 8337 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:17:07.695466 8337 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:17:07.695623 8337 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:17:07.699604 8337 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.254:40121
I20260923 23:17:07.699632 8404 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.254:40121 every 8 connection(s)
I20260923 23:17:07.699950 8337 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
I20260923 23:17:07.700423 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 8337
I20260923 23:17:07.700532 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/master-0/wal/instance
I20260923 23:17:07.700726 25487 external_mini_cluster.cc:1428] Setting key 293f6b063924398538031685a64d8b88
I20260923 23:17:07.700769 8405 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:17:07.703979 8405 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70625401bbe84b8faa08091c510ffafe: Bootstrap starting.
I20260923 23:17:07.704555 8405 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 70625401bbe84b8faa08091c510ffafe: Neither blocks nor log segments found. Creating new log.
I20260923 23:17:07.704866 8405 log.cc:824] T 00000000000000000000000000000000 P 70625401bbe84b8faa08091c510ffafe: Log is configured to *not* fsync() on all Append() calls
I20260923 23:17:07.705581 8405 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70625401bbe84b8faa08091c510ffafe: No bootstrap required, opened a new log
Sep 23 23:17:07 dist-test-slave-754n krb5kdc[8324](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205427, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
I20260923 23:17:07.707778 8405 raft_consensus.cc:348] T 00000000000000000000000000000000 P 70625401bbe84b8faa08091c510ffafe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70625401bbe84b8faa08091c510ffafe" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 40121 } }
I20260923 23:17:07.707988 8405 raft_consensus.cc:374] T 00000000000000000000000000000000 P 70625401bbe84b8faa08091c510ffafe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:17:07.708065 8405 raft_consensus.cc:729] T 00000000000000000000000000000000 P 70625401bbe84b8faa08091c510ffafe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70625401bbe84b8faa08091c510ffafe, State: Initialized, Role: FOLLOWER
I20260923 23:17:07.708201 8405 consensus_queue.cc:260] T 00000000000000000000000000000000 P 70625401bbe84b8faa08091c510ffafe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70625401bbe84b8faa08091c510ffafe" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 40121 } }
I20260923 23:17:07.708273 8405 raft_consensus.cc:388] T 00000000000000000000000000000000 P 70625401bbe84b8faa08091c510ffafe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 23:17:07.708295 8405 raft_consensus.cc:482] T 00000000000000000000000000000000 P 70625401bbe84b8faa08091c510ffafe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 23:17:07.708364 8405 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 70625401bbe84b8faa08091c510ffafe [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:17:07.709010 8405 raft_consensus.cc:504] T 00000000000000000000000000000000 P 70625401bbe84b8faa08091c510ffafe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70625401bbe84b8faa08091c510ffafe" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 40121 } }
I20260923 23:17:07.709138 8405 leader_election.cc:302] T 00000000000000000000000000000000 P 70625401bbe84b8faa08091c510ffafe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 70625401bbe84b8faa08091c510ffafe; no voters:
I20260923 23:17:07.709307 8405 leader_election.cc:288] T 00000000000000000000000000000000 P 70625401bbe84b8faa08091c510ffafe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 23:17:07.709350 8410 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 70625401bbe84b8faa08091c510ffafe [term 1 FOLLOWER]: Leader election won for term 1
I20260923 23:17:07.709558 8410 raft_consensus.cc:686] T 00000000000000000000000000000000 P 70625401bbe84b8faa08091c510ffafe [term 1 LEADER]: Becoming Leader. State: Replica: 70625401bbe84b8faa08091c510ffafe, State: Running, Role: LEADER
I20260923 23:17:07.709580 8405 sys_catalog.cc:564] T 00000000000000000000000000000000 P 70625401bbe84b8faa08091c510ffafe [sys.catalog]: configured and running, proceeding with master startup.
I20260923 23:17:07.709707 8410 consensus_queue.cc:237] T 00000000000000000000000000000000 P 70625401bbe84b8faa08091c510ffafe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70625401bbe84b8faa08091c510ffafe" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 40121 } }
I20260923 23:17:07.710968 8412 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70625401bbe84b8faa08091c510ffafe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "70625401bbe84b8faa08091c510ffafe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70625401bbe84b8faa08091c510ffafe" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 40121 } } }
I20260923 23:17:07.711122 8411 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70625401bbe84b8faa08091c510ffafe [sys.catalog]: SysCatalogTable state changed. Reason: New leader 70625401bbe84b8faa08091c510ffafe. Latest consensus state: current_term: 1 leader_uuid: "70625401bbe84b8faa08091c510ffafe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70625401bbe84b8faa08091c510ffafe" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 40121 } } }
I20260923 23:17:07.711251 8412 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70625401bbe84b8faa08091c510ffafe [sys.catalog]: This master's current role is: LEADER
I20260923 23:17:07.711267 8411 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70625401bbe84b8faa08091c510ffafe [sys.catalog]: This master's current role is: LEADER
W20260923 23:17:07.711498 8425 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 70625401bbe84b8faa08091c510ffafe: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 23:17:07.711617 8425 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 23:17:07.711712 8427 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 23:17:07.711898 8427 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 23:17:07.711966 8408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:17:07.701180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33952 (local address 127.24.227.254:40121)
0923 23:17:07.701375 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0923 23:17:07.701384 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 23:17:07.701412 (+ 28us) server_negotiation.cc:407] Connection header received
0923 23:17:07.701864 (+ 452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:17:07.701872 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:17:07.701931 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:17:07.702076 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:17:07.702631 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:17:07.704010 (+ 1379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:17:07.704787 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:17:07.705336 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:17:07.705466 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:17:07.708021 (+ 2555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:17:07.708025 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:17:07.708025 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:17:07.708052 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:17:07.710090 (+ 2038us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:17:07.710551 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:17:07.710554 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:17:07.710554 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:17:07.710688 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:17:07.710887 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:17:07.710888 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:17:07.710889 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:17:07.711178 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:17:07.711331 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0923 23:17:07.711611 (+ 280us) server_negotiation.cc:299] Negotiation successful
0923 23:17:07.711752 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":88,"threads_started":1}
I20260923 23:17:07.713203 8427 catalog_manager.cc:1347] Generated new cluster ID: f235dee3b5b04dd48ee9f905690e8ef1
I20260923 23:17:07.713253 8427 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 23:17:07.738329 8427 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 23:17:07.738824 8427 catalog_manager.cc:1504] Loading token signing keys...
I20260923 23:17:07.742726 8427 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 70625401bbe84b8faa08091c510ffafe: Generated new TSK 0
I20260923 23:17:07.742894 8427 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.227.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.227.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.227.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.193.keytab.
Entry for principal kudu/127.24.227.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.193.keytab.
I20260923 23:17:07.774703 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.227.193:0
--local_ip_for_outbound_sockets=127.24.227.193
--webserver_interface=127.24.227.193
--webserver_port=0
--tserver_master_addrs=127.24.227.254:40121
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.193.keytab
--principal=kudu/127.24.227.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:33557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:17:07.871186 8431 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:17:07.871380 8431 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:17:07.871414 8431 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:17:07.872603 8431 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:17:07.872668 8431 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:17:07.872711 8431 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.227.193
I20260923 23:17:07.874198 8431 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:33557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.193.keytab
--principal=kudu/127.24.227.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.227.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.227.193
--webserver_port=0
--tserver_master_addrs=127.24.227.254:40121
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8431
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.227.193
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:17:07.874426 8431 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:17:07.874635 8431 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:17:07.879495 8436 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:17:07.879469 8439 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:17:07.879480 8437 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:17:07.879889 8431 server_base.cc:1034] running on GCE node
I20260923 23:17:07.880070 8431 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:17:07.880262 8431 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:17:07.881421 8431 hybrid_clock.cc:648] HybridClock initialized: now 1790205427881411 us; error 38 us; skew 500 ppm
Sep 23 23:17:07 dist-test-slave-754n krb5kdc[8324](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205427, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:17:07.883929 8431 init.cc:377] Logged in from keytab as kudu/127.24.227.193@KRBTEST.COM (short username kudu)
I20260923 23:17:07.884327 8431 webserver.cc:460] Webserver started at http://127.24.227.193:37869/ using document root <none> and password file <none>
I20260923 23:17:07.884512 8431 fs_manager.cc:359] Metadata directory not provided
I20260923 23:17:07.884549 8431 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:17:07.884639 8431 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:17:07.885485 8431 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-0/data/instance:
uuid: "9a094e6b7a64477fb4cda66abefcbdfd"
format_stamp: "Formatted at 2026-09-23 23:17:07 on dist-test-slave-754n"
server_key: "5fb09fd0f1d54d0105773590ccf76e2f"
server_key_iv: "f3b3e0fe422ee48e48d46a1ec2389715"
server_key_version: "encryptionkey@0"
I20260923 23:17:07.885797 8431 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-0/wal/instance:
uuid: "9a094e6b7a64477fb4cda66abefcbdfd"
format_stamp: "Formatted at 2026-09-23 23:17:07 on dist-test-slave-754n"
server_key: "5fb09fd0f1d54d0105773590ccf76e2f"
server_key_iv: "f3b3e0fe422ee48e48d46a1ec2389715"
server_key_version: "encryptionkey@0"
I20260923 23:17:07.886956 8431 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 23:17:07.887641 8446 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:17:07.887789 8431 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 23:17:07.887904 8431 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-0/wal
uuid: "9a094e6b7a64477fb4cda66abefcbdfd"
format_stamp: "Formatted at 2026-09-23 23:17:07 on dist-test-slave-754n"
server_key: "5fb09fd0f1d54d0105773590ccf76e2f"
server_key_iv: "f3b3e0fe422ee48e48d46a1ec2389715"
server_key_version: "encryptionkey@0"
I20260923 23:17:07.887993 8431 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:17:07.908342 8431 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:17:07.910275 8431 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:17:07.910429 8431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:17:07.910658 8431 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 23:17:07.911008 8431 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 23:17:07.911043 8431 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 23:17:07.911072 8431 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 23:17:07.911104 8431 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 23:17:07.916675 8431 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.193:46547
I20260923 23:17:07.916734 8559 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.193:46547 every 8 connection(s)
I20260923 23:17:07.917078 8431 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-0/data/info.pb
I20260923 23:17:07.919591 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 8431
I20260923 23:17:07.919691 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-0/wal/instance
I20260923 23:17:07.919924 25487 external_mini_cluster.cc:1428] Setting key 759ab5fadbff672b2f5d1fbae6dd4405
Sep 23 23:17:07 dist-test-slave-754n krb5kdc[8324](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205427, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.193@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
I20260923 23:17:07.926218 8408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:17:07.917761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.193:49163 (local address 127.24.227.254:40121)
0923 23:17:07.917849 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0923 23:17:07.917852 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 23:17:07.918060 (+ 208us) server_negotiation.cc:407] Connection header received
0923 23:17:07.918491 (+ 431us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:17:07.918492 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:17:07.918502 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:17:07.918553 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:17:07.919369 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:17:07.919987 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:17:07.921120 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:17:07.921345 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:17:07.921422 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:17:07.923439 (+ 2017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:17:07.923440 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:17:07.923441 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:17:07.923447 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:17:07.924876 (+ 1429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:17:07.925273 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:17:07.925274 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:17:07.925274 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:17:07.925332 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:17:07.925593 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:17:07.925594 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:17:07.925594 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:17:07.925703 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:17:07.925776 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0923 23:17:07.926047 (+ 271us) server_negotiation.cc:299] Negotiation successful
0923 23:17:07.926134 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260923 23:17:07.926385 8562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:17:07.917735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:40121 (local address 127.24.227.193:49163)
0923 23:17:07.917939 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0923 23:17:07.917972 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0923 23:17:07.918409 (+ 437us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:17:07.918619 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:17:07.918622 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:17:07.919131 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:17:07.919296 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:17:07.919300 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:17:07.920071 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:17:07.920072 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:17:07.921014 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:17:07.921018 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:17:07.921433 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:17:07.921435 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:17:07.921566 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:17:07.922062 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:17:07.922071 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:17:07.923342 (+ 1271us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:17:07.924957 (+ 1615us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:17:07.924964 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:17:07.924966 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:17:07.925184 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:17:07.925408 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:17:07.925409 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:17:07.925409 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:17:07.925513 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:17:07.925782 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:17:07.925789 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:17:07.925918 (+ 129us) client_negotiation.cc:769] Sending connection context
0923 23:17:07.926015 (+ 97us) client_negotiation.cc:240] Negotiation successful
0923 23:17:07.926164 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":118,"threads_started":1}
I20260923 23:17:07.926880 8560 heartbeater.cc:347] Connected to a master server at 127.24.227.254:40121
I20260923 23:17:07.926970 8560 heartbeater.cc:464] Registering TS with master...
I20260923 23:17:07.927160 8560 heartbeater.cc:511] Master 127.24.227.254:40121 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.24.227.194@KRBTEST.COM; defaulting to no policy
I20260923 23:17:07.927647 8369 ts_manager.cc:194] Registered new tserver with Master: 9a094e6b7a64477fb4cda66abefcbdfd (127.24.227.193:46547)
I20260923 23:17:07.928359 8369 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.227.193@KRBTEST.COM'} at 127.24.227.193:49163
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.227.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.227.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.194.keytab.
Entry for principal kudu/127.24.227.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.194.keytab.
I20260923 23:17:07.942667 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.227.194:0
--local_ip_for_outbound_sockets=127.24.227.194
--webserver_interface=127.24.227.194
--webserver_port=0
--tserver_master_addrs=127.24.227.254:40121
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.194.keytab
--principal=kudu/127.24.227.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:33557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:17:08.021363 8565 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:17:08.021586 8565 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:17:08.021625 8565 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:17:08.022779 8565 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:17:08.022842 8565 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:17:08.022886 8565 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.227.194
I20260923 23:17:08.024294 8565 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:33557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.194.keytab
--principal=kudu/127.24.227.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.227.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.227.194
--webserver_port=0
--tserver_master_addrs=127.24.227.254:40121
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8565
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.227.194
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:17:08.024541 8565 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:17:08.024761 8565 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:17:08.029352 8573 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:17:08.029385 8571 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:17:08.029376 8570 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:17:08.029522 8565 server_base.cc:1034] running on GCE node
I20260923 23:17:08.029726 8565 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:17:08.029976 8565 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:17:08.031123 8565 hybrid_clock.cc:648] HybridClock initialized: now 1790205428031111 us; error 28 us; skew 500 ppm
Sep 23 23:17:08 dist-test-slave-754n krb5kdc[8324](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205428, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:17:08.033520 8565 init.cc:377] Logged in from keytab as kudu/127.24.227.194@KRBTEST.COM (short username kudu)
I20260923 23:17:08.033938 8565 webserver.cc:460] Webserver started at http://127.24.227.194:34375/ using document root <none> and password file <none>
I20260923 23:17:08.034176 8565 fs_manager.cc:359] Metadata directory not provided
I20260923 23:17:08.034245 8565 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:17:08.034392 8565 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:17:08.035416 8565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-1/data/instance:
uuid: "5ac6b50428b346a1b5604b42baf02cee"
format_stamp: "Formatted at 2026-09-23 23:17:08 on dist-test-slave-754n"
server_key: "2cc6b3f7ca7a86612a83fcbd7ba6a005"
server_key_iv: "463e05b2ac27eb65958963ee9210a15c"
server_key_version: "encryptionkey@0"
I20260923 23:17:08.035769 8565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-1/wal/instance:
uuid: "5ac6b50428b346a1b5604b42baf02cee"
format_stamp: "Formatted at 2026-09-23 23:17:08 on dist-test-slave-754n"
server_key: "2cc6b3f7ca7a86612a83fcbd7ba6a005"
server_key_iv: "463e05b2ac27eb65958963ee9210a15c"
server_key_version: "encryptionkey@0"
I20260923 23:17:08.037158 8565 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 23:17:08.037842 8580 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:17:08.038025 8565 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 23:17:08.038116 8565 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-1/wal
uuid: "5ac6b50428b346a1b5604b42baf02cee"
format_stamp: "Formatted at 2026-09-23 23:17:08 on dist-test-slave-754n"
server_key: "2cc6b3f7ca7a86612a83fcbd7ba6a005"
server_key_iv: "463e05b2ac27eb65958963ee9210a15c"
server_key_version: "encryptionkey@0"
I20260923 23:17:08.038194 8565 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:17:08.054890 8565 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:17:08.057083 8565 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:17:08.057240 8565 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:17:08.057466 8565 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 23:17:08.057768 8565 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 23:17:08.057822 8565 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 23:17:08.057873 8565 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 23:17:08.057901 8565 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 23:17:08.063683 8565 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.194:36765
I20260923 23:17:08.063730 8693 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.194:36765 every 8 connection(s)
I20260923 23:17:08.064052 8565 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-1/data/info.pb
I20260923 23:17:08.066910 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 8565
I20260923 23:17:08.066994 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-1/wal/instance
I20260923 23:17:08.067234 25487 external_mini_cluster.cc:1428] Setting key 06ec99dde050ac4b00a9d697518c8a2f
Sep 23 23:17:08 dist-test-slave-754n krb5kdc[8324](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205428, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.194@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
I20260923 23:17:08.073625 8408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:17:08.064737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.194:59181 (local address 127.24.227.254:40121)
0923 23:17:08.064836 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0923 23:17:08.064839 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 23:17:08.065017 (+ 178us) server_negotiation.cc:407] Connection header received
0923 23:17:08.065539 (+ 522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:17:08.065540 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:17:08.065560 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:17:08.065607 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:17:08.066520 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:17:08.067092 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:17:08.068313 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:17:08.068519 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:17:08.068605 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:17:08.070802 (+ 2197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:17:08.070803 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:17:08.070804 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:17:08.070810 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:17:08.072254 (+ 1444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:17:08.072680 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:17:08.072681 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:17:08.072682 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:17:08.072761 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:17:08.073024 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:17:08.073024 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:17:08.073025 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:17:08.073129 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:17:08.073196 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0923 23:17:08.073460 (+ 264us) server_negotiation.cc:299] Negotiation successful
0923 23:17:08.073535 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260923 23:17:08.073853 8696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:17:08.064696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:40121 (local address 127.24.227.194:59181)
0923 23:17:08.064880 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0923 23:17:08.064920 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0923 23:17:08.065435 (+ 515us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:17:08.065703 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:17:08.065707 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:17:08.066246 (+ 539us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:17:08.066422 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:17:08.066427 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:17:08.067169 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:17:08.067170 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:17:08.068230 (+ 1060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:17:08.068232 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:17:08.068616 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:17:08.068617 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:17:08.068746 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:17:08.069232 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:17:08.069243 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:17:08.070704 (+ 1461us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:17:08.072350 (+ 1646us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:17:08.072357 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:17:08.072359 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:17:08.072588 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:17:08.072833 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:17:08.072834 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:17:08.072834 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:17:08.072937 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:17:08.073200 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:17:08.073207 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:17:08.073337 (+ 130us) client_negotiation.cc:769] Sending connection context
0923 23:17:08.073440 (+ 103us) client_negotiation.cc:240] Negotiation successful
0923 23:17:08.073595 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":93,"threads_started":1}
I20260923 23:17:08.074285 8694 heartbeater.cc:347] Connected to a master server at 127.24.227.254:40121
I20260923 23:17:08.074386 8694 heartbeater.cc:464] Registering TS with master...
I20260923 23:17:08.074568 8694 heartbeater.cc:511] Master 127.24.227.254:40121 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.24.227.195@KRBTEST.COM; defaulting to no policy
I20260923 23:17:08.074911 8369 ts_manager.cc:194] Registered new tserver with Master: 5ac6b50428b346a1b5604b42baf02cee (127.24.227.194:36765)
I20260923 23:17:08.075301 8369 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.227.194@KRBTEST.COM'} at 127.24.227.194:59181
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.227.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.227.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.195.keytab.
Entry for principal kudu/127.24.227.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.195.keytab.
I20260923 23:17:08.090678 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.227.195:0
--local_ip_for_outbound_sockets=127.24.227.195
--webserver_interface=127.24.227.195
--webserver_port=0
--tserver_master_addrs=127.24.227.254:40121
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.195.keytab
--principal=kudu/127.24.227.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:33557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:17:08.164865 8699 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:17:08.165043 8699 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:17:08.165066 8699 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:17:08.166112 8699 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:17:08.166144 8699 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:17:08.166168 8699 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.227.195
I20260923 23:17:08.167493 8699 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:33557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.195.keytab
--principal=kudu/127.24.227.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.227.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.227.195
--webserver_port=0
--tserver_master_addrs=127.24.227.254:40121
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8699
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.227.195
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:17:08.167730 8699 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:17:08.167972 8699 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:17:08.172448 8705 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:17:08.172456 8704 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:17:08.172458 8707 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:17:08.172780 8699 server_base.cc:1034] running on GCE node
I20260923 23:17:08.173030 8699 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:17:08.173290 8699 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:17:08.174427 8699 hybrid_clock.cc:648] HybridClock initialized: now 1790205428174416 us; error 28 us; skew 500 ppm
Sep 23 23:17:08 dist-test-slave-754n krb5kdc[8324](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205428, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:17:08.177060 8699 init.cc:377] Logged in from keytab as kudu/127.24.227.195@KRBTEST.COM (short username kudu)
I20260923 23:17:08.177409 8699 webserver.cc:460] Webserver started at http://127.24.227.195:38179/ using document root <none> and password file <none>
I20260923 23:17:08.177578 8699 fs_manager.cc:359] Metadata directory not provided
I20260923 23:17:08.177618 8699 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:17:08.177698 8699 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:17:08.178602 8699 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-2/data/instance:
uuid: "b8a62c956c3e46baabad006ee1791b9d"
format_stamp: "Formatted at 2026-09-23 23:17:08 on dist-test-slave-754n"
server_key: "63c6ad468db4723a9a3d64de89590129"
server_key_iv: "4728729873838112e0e61078612ec59c"
server_key_version: "encryptionkey@0"
I20260923 23:17:08.178892 8699 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-2/wal/instance:
uuid: "b8a62c956c3e46baabad006ee1791b9d"
format_stamp: "Formatted at 2026-09-23 23:17:08 on dist-test-slave-754n"
server_key: "63c6ad468db4723a9a3d64de89590129"
server_key_iv: "4728729873838112e0e61078612ec59c"
server_key_version: "encryptionkey@0"
I20260923 23:17:08.180157 8699 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 23:17:08.180855 8714 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:17:08.181030 8699 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 23:17:08.181121 8699 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-2/wal
uuid: "b8a62c956c3e46baabad006ee1791b9d"
format_stamp: "Formatted at 2026-09-23 23:17:08 on dist-test-slave-754n"
server_key: "63c6ad468db4723a9a3d64de89590129"
server_key_iv: "4728729873838112e0e61078612ec59c"
server_key_version: "encryptionkey@0"
I20260923 23:17:08.181200 8699 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:17:08.197335 8699 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:17:08.199553 8699 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:17:08.199700 8699 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:17:08.199958 8699 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 23:17:08.200241 8699 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 23:17:08.200292 8699 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 23:17:08.200330 8699 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 23:17:08.200356 8699 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 23:17:08.205715 8699 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.195:35361
I20260923 23:17:08.205775 8827 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.195:35361 every 8 connection(s)
I20260923 23:17:08.206073 8699 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-2/data/info.pb
Sep 23 23:17:08 dist-test-slave-754n krb5kdc[8324](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205428, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.195@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
I20260923 23:17:08.214548 8408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:17:08.206781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.195:45401 (local address 127.24.227.254:40121)
0923 23:17:08.206841 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0923 23:17:08.206843 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:17:08.207002 (+ 159us) server_negotiation.cc:407] Connection header received
0923 23:17:08.207436 (+ 434us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:17:08.207437 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:17:08.207446 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:17:08.207499 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:17:08.208385 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:17:08.208857 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:17:08.209914 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:17:08.210101 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:17:08.210152 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:17:08.212452 (+ 2300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:17:08.212453 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:17:08.212453 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:17:08.212459 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:17:08.213532 (+ 1073us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:17:08.213858 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:17:08.213859 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:17:08.213859 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:17:08.213905 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:17:08.214099 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:17:08.214099 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:17:08.214100 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:17:08.214187 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:17:08.214233 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0923 23:17:08.214435 (+ 202us) server_negotiation.cc:299] Negotiation successful
0923 23:17:08.214489 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260923 23:17:08.214752 8830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:17:08.206753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:40121 (local address 127.24.227.195:45401)
0923 23:17:08.206896 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0923 23:17:08.206933 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0923 23:17:08.207360 (+ 427us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:17:08.207548 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:17:08.207551 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:17:08.208139 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:17:08.208311 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:17:08.208316 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:17:08.208928 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:17:08.208929 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:17:08.209841 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:17:08.209844 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:17:08.210171 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:17:08.210172 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:17:08.210408 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:17:08.210887 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:17:08.210899 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:17:08.212374 (+ 1475us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:17:08.213596 (+ 1222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:17:08.213602 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:17:08.213603 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:17:08.213795 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:17:08.213959 (+ 164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:17:08.213960 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:17:08.213960 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:17:08.214047 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:17:08.214238 (+ 191us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:17:08.214242 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:17:08.214337 (+ 95us) client_negotiation.cc:769] Sending connection context
0923 23:17:08.214409 (+ 72us) client_negotiation.cc:240] Negotiation successful
0923 23:17:08.214510 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":79,"threads_started":1}
I20260923 23:17:08.214828 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 8699
I20260923 23:17:08.214906 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-2/wal/instance
I20260923 23:17:08.215080 25487 external_mini_cluster.cc:1428] Setting key 49ec876ca79e5810b0174ef4a3732b03
I20260923 23:17:08.215179 8828 heartbeater.cc:347] Connected to a master server at 127.24.227.254:40121
I20260923 23:17:08.215268 8828 heartbeater.cc:464] Registering TS with master...
I20260923 23:17:08.215438 8828 heartbeater.cc:511] Master 127.24.227.254:40121 requested a full tablet report, sending...
I20260923 23:17:08.215755 8368 ts_manager.cc:194] Registered new tserver with Master: b8a62c956c3e46baabad006ee1791b9d (127.24.227.195:35361)
I20260923 23:17:08.216136 8368 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.227.195@KRBTEST.COM'} at 127.24.227.195:45401
I20260923 23:17:08.226759 25487 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 23:17:08.233484 8408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:17:08.227998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33960 (local address 127.24.227.254:40121)
0923 23:17:08.228028 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0923 23:17:08.228030 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:17:08.228155 (+ 125us) server_negotiation.cc:407] Connection header received
0923 23:17:08.228226 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:17:08.228227 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:17:08.228236 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:17:08.228283 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:17:08.228825 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:17:08.229285 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:17:08.229999 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:17:08.230195 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:17:08.230264 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:17:08.231234 (+ 970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:17:08.231235 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:17:08.231236 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:17:08.231242 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:17:08.232375 (+ 1133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:17:08.232736 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:17:08.232737 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:17:08.232737 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:17:08.232791 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:17:08.232983 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:17:08.232984 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:17:08.232984 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:17:08.233114 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:17:08.233213 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0923 23:17:08.233324 (+ 111us) server_negotiation.cc:299] Negotiation successful
0923 23:17:08.233395 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260923 23:17:08.235106 8368 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33960:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 23:17:08.236635 8368 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 23:17:08.242735 8837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:17:08.238566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.195:35361 (local address 127.0.0.1:34070)
0923 23:17:08.238701 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0923 23:17:08.238733 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0923 23:17:08.238789 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:17:08.239104 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:17:08.239106 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:17:08.239109 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:17:08.239182 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:17:08.239183 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:17:08.240285 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:17:08.240286 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:17:08.241537 (+ 1251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:17:08.241539 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:17:08.242428 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:17:08.242429 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:17:08.242516 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:17:08.242576 (+ 60us) client_negotiation.cc:769] Sending connection context
0923 23:17:08.242613 (+ 37us) client_negotiation.cc:240] Negotiation successful
0923 23:17:08.242659 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"mutex_wait_us":55,"thread_start_us":52,"threads_started":1}
I20260923 23:17:08.242759 8839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:17:08.238708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34070 (local address 127.24.227.195:35361)
0923 23:17:08.238948 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0923 23:17:08.238951 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 23:17:08.238959 (+ 8us) server_negotiation.cc:407] Connection header received
0923 23:17:08.238987 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:17:08.238989 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:17:08.239008 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:17:08.239096 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:17:08.239265 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:17:08.240187 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:17:08.241609 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:17:08.242290 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:17:08.242542 (+ 252us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:17:08.242603 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0923 23:17:08.242640 (+ 37us) server_negotiation.cc:299] Negotiation successful
0923 23:17:08.242680 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":218,"thread_start_us":65,"threads_started":1}
I20260923 23:17:08.242974 8840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:17:08.238787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42714 (local address 127.24.227.194:36765)
0923 23:17:08.238916 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0923 23:17:08.238920 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 23:17:08.239202 (+ 282us) server_negotiation.cc:407] Connection header received
0923 23:17:08.239248 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:17:08.239250 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:17:08.239265 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:17:08.239331 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:17:08.239562 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:17:08.240629 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:17:08.241833 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:17:08.242297 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:17:08.242406 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:17:08.242459 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0923 23:17:08.242712 (+ 253us) server_negotiation.cc:299] Negotiation successful
0923 23:17:08.242901 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":69,"threads_started":1}
I20260923 23:17:08.243142 8838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:17:08.238599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.194:36765 (local address 127.0.0.1:42714)
0923 23:17:08.239076 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0923 23:17:08.239083 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0923 23:17:08.239123 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:17:08.239434 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:17:08.239435 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:17:08.239437 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:17:08.239486 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:17:08.239488 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:17:08.240721 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:17:08.240722 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:17:08.241764 (+ 1042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:17:08.241766 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:17:08.242416 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:17:08.242417 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:17:08.242496 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:17:08.242509 (+ 13us) client_negotiation.cc:769] Sending connection context
0923 23:17:08.242576 (+ 67us) client_negotiation.cc:240] Negotiation successful
0923 23:17:08.242802 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"mutex_wait_us":3,"thread_start_us":57,"threads_started":1}
I20260923 23:17:08.243680 8836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:17:08.238576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.193:46547 (local address 127.0.0.1:35392)
0923 23:17:08.238938 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0923 23:17:08.238947 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 23:17:08.238994 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:17:08.239207 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:17:08.239208 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:17:08.239210 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:17:08.239253 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:17:08.239254 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:17:08.240664 (+ 1410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:17:08.240666 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:17:08.241613 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:17:08.241615 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:17:08.242454 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:17:08.242455 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:17:08.242516 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:17:08.242554 (+ 38us) client_negotiation.cc:769] Sending connection context
0923 23:17:08.242587 (+ 33us) client_negotiation.cc:240] Negotiation successful
0923 23:17:08.242635 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":351,"mutex_wait_us":3,"thread_start_us":44,"threads_started":1}
I20260923 23:17:08.243710 8628 tablet_service.cc:1467] Processing CreateTablet for tablet af9f9081dffd439a8685396e3403db89 (DEFAULT_TABLE table=test-table [id=52fe3a37f4c440298129b1a8b6726b11]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 23:17:08.243714 8762 tablet_service.cc:1467] Processing CreateTablet for tablet af9f9081dffd439a8685396e3403db89 (DEFAULT_TABLE table=test-table [id=52fe3a37f4c440298129b1a8b6726b11]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 23:17:08.244069 8628 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af9f9081dffd439a8685396e3403db89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:17:08.244062 8762 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af9f9081dffd439a8685396e3403db89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:17:08.244117 8841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:17:08.238794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35392 (local address 127.24.227.193:46547)
0923 23:17:08.238932 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0923 23:17:08.238936 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 23:17:08.238985 (+ 49us) server_negotiation.cc:407] Connection header received
0923 23:17:08.239063 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:17:08.239065 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:17:08.239080 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:17:08.239202 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:17:08.239311 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:17:08.240497 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:17:08.241705 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:17:08.242348 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:17:08.243871 (+ 1523us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:17:08.243952 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0923 23:17:08.243977 (+ 25us) server_negotiation.cc:299] Negotiation successful
0923 23:17:08.244028 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":77,"threads_started":1}
I20260923 23:17:08.244630 8494 tablet_service.cc:1467] Processing CreateTablet for tablet af9f9081dffd439a8685396e3403db89 (DEFAULT_TABLE table=test-table [id=52fe3a37f4c440298129b1a8b6726b11]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 23:17:08.244925 8494 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af9f9081dffd439a8685396e3403db89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:17:08.247205 8843 tablet_bootstrap.cc:492] T af9f9081dffd439a8685396e3403db89 P b8a62c956c3e46baabad006ee1791b9d: Bootstrap starting.
I20260923 23:17:08.247524 8842 tablet_bootstrap.cc:492] T af9f9081dffd439a8685396e3403db89 P 5ac6b50428b346a1b5604b42baf02cee: Bootstrap starting.
I20260923 23:17:08.247939 8843 tablet_bootstrap.cc:654] T af9f9081dffd439a8685396e3403db89 P b8a62c956c3e46baabad006ee1791b9d: Neither blocks nor log segments found. Creating new log.
I20260923 23:17:08.248066 8844 tablet_bootstrap.cc:492] T af9f9081dffd439a8685396e3403db89 P 9a094e6b7a64477fb4cda66abefcbdfd: Bootstrap starting.
I20260923 23:17:08.248152 8842 tablet_bootstrap.cc:654] T af9f9081dffd439a8685396e3403db89 P 5ac6b50428b346a1b5604b42baf02cee: Neither blocks nor log segments found. Creating new log.
I20260923 23:17:08.248572 8843 log.cc:824] T af9f9081dffd439a8685396e3403db89 P b8a62c956c3e46baabad006ee1791b9d: Log is configured to *not* fsync() on all Append() calls
I20260923 23:17:08.248664 8842 log.cc:824] T af9f9081dffd439a8685396e3403db89 P 5ac6b50428b346a1b5604b42baf02cee: Log is configured to *not* fsync() on all Append() calls
I20260923 23:17:08.248860 8844 tablet_bootstrap.cc:654] T af9f9081dffd439a8685396e3403db89 P 9a094e6b7a64477fb4cda66abefcbdfd: Neither blocks nor log segments found. Creating new log.
I20260923 23:17:08.249313 8843 tablet_bootstrap.cc:492] T af9f9081dffd439a8685396e3403db89 P b8a62c956c3e46baabad006ee1791b9d: No bootstrap required, opened a new log
I20260923 23:17:08.249383 8843 ts_tablet_manager.cc:1397] T af9f9081dffd439a8685396e3403db89 P b8a62c956c3e46baabad006ee1791b9d: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260923 23:17:08.249400 8842 tablet_bootstrap.cc:492] T af9f9081dffd439a8685396e3403db89 P 5ac6b50428b346a1b5604b42baf02cee: No bootstrap required, opened a new log
I20260923 23:17:08.249449 8842 ts_tablet_manager.cc:1397] T af9f9081dffd439a8685396e3403db89 P 5ac6b50428b346a1b5604b42baf02cee: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 23:17:08.249508 8844 log.cc:824] T af9f9081dffd439a8685396e3403db89 P 9a094e6b7a64477fb4cda66abefcbdfd: Log is configured to *not* fsync() on all Append() calls
I20260923 23:17:08.250384 8844 tablet_bootstrap.cc:492] T af9f9081dffd439a8685396e3403db89 P 9a094e6b7a64477fb4cda66abefcbdfd: No bootstrap required, opened a new log
I20260923 23:17:08.250504 8844 ts_tablet_manager.cc:1397] T af9f9081dffd439a8685396e3403db89 P 9a094e6b7a64477fb4cda66abefcbdfd: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260923 23:17:08.251284 8842 raft_consensus.cc:348] T af9f9081dffd439a8685396e3403db89 P 5ac6b50428b346a1b5604b42baf02cee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8a62c956c3e46baabad006ee1791b9d" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 35361 } } peers { permanent_uuid: "9a094e6b7a64477fb4cda66abefcbdfd" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 46547 } } peers { permanent_uuid: "5ac6b50428b346a1b5604b42baf02cee" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 36765 } }
I20260923 23:17:08.251284 8843 raft_consensus.cc:348] T af9f9081dffd439a8685396e3403db89 P b8a62c956c3e46baabad006ee1791b9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8a62c956c3e46baabad006ee1791b9d" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 35361 } } peers { permanent_uuid: "9a094e6b7a64477fb4cda66abefcbdfd" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 46547 } } peers { permanent_uuid: "5ac6b50428b346a1b5604b42baf02cee" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 36765 } }
I20260923 23:17:08.251458 8842 raft_consensus.cc:374] T af9f9081dffd439a8685396e3403db89 P 5ac6b50428b346a1b5604b42baf02cee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:17:08.251458 8843 raft_consensus.cc:374] T af9f9081dffd439a8685396e3403db89 P b8a62c956c3e46baabad006ee1791b9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:17:08.251509 8842 raft_consensus.cc:729] T af9f9081dffd439a8685396e3403db89 P 5ac6b50428b346a1b5604b42baf02cee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ac6b50428b346a1b5604b42baf02cee, State: Initialized, Role: FOLLOWER
I20260923 23:17:08.251509 8843 raft_consensus.cc:729] T af9f9081dffd439a8685396e3403db89 P b8a62c956c3e46baabad006ee1791b9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8a62c956c3e46baabad006ee1791b9d, State: Initialized, Role: FOLLOWER
I20260923 23:17:08.251646 8843 consensus_queue.cc:260] T af9f9081dffd439a8685396e3403db89 P b8a62c956c3e46baabad006ee1791b9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8a62c956c3e46baabad006ee1791b9d" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 35361 } } peers { permanent_uuid: "9a094e6b7a64477fb4cda66abefcbdfd" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 46547 } } peers { permanent_uuid: "5ac6b50428b346a1b5604b42baf02cee" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 36765 } }
I20260923 23:17:08.251646 8842 consensus_queue.cc:260] T af9f9081dffd439a8685396e3403db89 P 5ac6b50428b346a1b5604b42baf02cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8a62c956c3e46baabad006ee1791b9d" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 35361 } } peers { permanent_uuid: "9a094e6b7a64477fb4cda66abefcbdfd" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 46547 } } peers { permanent_uuid: "5ac6b50428b346a1b5604b42baf02cee" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 36765 } }
I20260923 23:17:08.251940 8843 ts_tablet_manager.cc:1428] T af9f9081dffd439a8685396e3403db89 P b8a62c956c3e46baabad006ee1791b9d: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260923 23:17:08.251940 8842 ts_tablet_manager.cc:1428] T af9f9081dffd439a8685396e3403db89 P 5ac6b50428b346a1b5604b42baf02cee: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260923 23:17:08.252125 8694 heartbeater.cc:503] Master 127.24.227.254:40121 was elected leader, sending a full tablet report...
I20260923 23:17:08.252137 8828 heartbeater.cc:503] Master 127.24.227.254:40121 was elected leader, sending a full tablet report...
I20260923 23:17:08.252346 8844 raft_consensus.cc:348] T af9f9081dffd439a8685396e3403db89 P 9a094e6b7a64477fb4cda66abefcbdfd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8a62c956c3e46baabad006ee1791b9d" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 35361 } } peers { permanent_uuid: "9a094e6b7a64477fb4cda66abefcbdfd" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 46547 } } peers { permanent_uuid: "5ac6b50428b346a1b5604b42baf02cee" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 36765 } }
I20260923 23:17:08.252563 8844 raft_consensus.cc:374] T af9f9081dffd439a8685396e3403db89 P 9a094e6b7a64477fb4cda66abefcbdfd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:17:08.252653 8844 raft_consensus.cc:729] T af9f9081dffd439a8685396e3403db89 P 9a094e6b7a64477fb4cda66abefcbdfd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a094e6b7a64477fb4cda66abefcbdfd, State: Initialized, Role: FOLLOWER
I20260923 23:17:08.252789 8844 consensus_queue.cc:260] T af9f9081dffd439a8685396e3403db89 P 9a094e6b7a64477fb4cda66abefcbdfd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8a62c956c3e46baabad006ee1791b9d" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 35361 } } peers { permanent_uuid: "9a094e6b7a64477fb4cda66abefcbdfd" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 46547 } } peers { permanent_uuid: "5ac6b50428b346a1b5604b42baf02cee" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 36765 } }
I20260923 23:17:08.253063 8844 ts_tablet_manager.cc:1428] T af9f9081dffd439a8685396e3403db89 P 9a094e6b7a64477fb4cda66abefcbdfd: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260923 23:17:08.253176 8560 heartbeater.cc:503] Master 127.24.227.254:40121 was elected leader, sending a full tablet report...
I20260923 23:17:08.254880 8848 raft_consensus.cc:482] T af9f9081dffd439a8685396e3403db89 P b8a62c956c3e46baabad006ee1791b9d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 23:17:08.254976 8848 raft_consensus.cc:504] T af9f9081dffd439a8685396e3403db89 P b8a62c956c3e46baabad006ee1791b9d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8a62c956c3e46baabad006ee1791b9d" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 35361 } } peers { permanent_uuid: "9a094e6b7a64477fb4cda66abefcbdfd" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 46547 } } peers { permanent_uuid: "5ac6b50428b346a1b5604b42baf02cee" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 36765 } }
I20260923 23:17:08.255266 8848 leader_election.cc:288] T af9f9081dffd439a8685396e3403db89 P b8a62c956c3e46baabad006ee1791b9d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9a094e6b7a64477fb4cda66abefcbdfd (127.24.227.193:46547), 5ac6b50428b346a1b5604b42baf02cee (127.24.227.194:36765)
I20260923 23:17:08.258785 8841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:17:08.255418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.195:33285 (local address 127.24.227.193:46547)
0923 23:17:08.255452 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0923 23:17:08.255454 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:17:08.255462 (+ 8us) server_negotiation.cc:407] Connection header received
0923 23:17:08.255524 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:17:08.255525 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:17:08.255535 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:17:08.255592 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:17:08.255763 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:17:08.256504 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:17:08.257915 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:17:08.258472 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:17:08.258561 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:17:08.258568 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0923 23:17:08.258697 (+ 129us) server_negotiation.cc:299] Negotiation successful
0923 23:17:08.258728 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260923 23:17:08.258785 8840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:17:08.255576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.195:35459 (local address 127.24.227.194:36765)
0923 23:17:08.255611 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0923 23:17:08.255613 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:17:08.255619 (+ 6us) server_negotiation.cc:407] Connection header received
0923 23:17:08.255641 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:17:08.255642 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:17:08.255652 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:17:08.255697 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:17:08.255819 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:17:08.256509 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:17:08.257915 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:17:08.258472 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:17:08.258546 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:17:08.258554 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0923 23:17:08.258697 (+ 143us) server_negotiation.cc:299] Negotiation successful
0923 23:17:08.258728 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260923 23:17:08.258807 8830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:17:08.255368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.193:46547 (local address 127.24.227.195:33285)
0923 23:17:08.255404 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0923 23:17:08.255416 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 23:17:08.255458 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:17:08.255634 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:17:08.255636 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:17:08.255637 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:17:08.255691 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:17:08.255692 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:17:08.256589 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:17:08.256590 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:17:08.257826 (+ 1236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:17:08.257829 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:17:08.258553 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:17:08.258554 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:17:08.258628 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:17:08.258630 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 23:17:08.258681 (+ 51us) client_negotiation.cc:240] Negotiation successful
0923 23:17:08.258727 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20,"mutex_wait_us":65}
I20260923 23:17:08.258817 8851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:17:08.255405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.194:36765 (local address 127.24.227.195:35459)
0923 23:17:08.255534 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0923 23:17:08.255553 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0923 23:17:08.255595 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:17:08.255700 (+ 105us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:17:08.255701 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:17:08.255702 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:17:08.255759 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:17:08.255760 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:17:08.256589 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:17:08.256590 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:17:08.257832 (+ 1242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:17:08.257833 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:17:08.258559 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:17:08.258559 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:17:08.258624 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:17:08.258627 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 23:17:08.258681 (+ 54us) client_negotiation.cc:240] Negotiation successful
0923 23:17:08.258725 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"mutex_wait_us":33,"thread_start_us":86,"threads_started":1}
I20260923 23:17:08.259092 8514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af9f9081dffd439a8685396e3403db89" candidate_uuid: "b8a62c956c3e46baabad006ee1791b9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a094e6b7a64477fb4cda66abefcbdfd" is_pre_election: true
I20260923 23:17:08.259205 8514 raft_consensus.cc:2454] T af9f9081dffd439a8685396e3403db89 P 9a094e6b7a64477fb4cda66abefcbdfd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8a62c956c3e46baabad006ee1791b9d in term 0.
I20260923 23:17:08.259184 8648 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af9f9081dffd439a8685396e3403db89" candidate_uuid: "b8a62c956c3e46baabad006ee1791b9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ac6b50428b346a1b5604b42baf02cee" is_pre_election: true
I20260923 23:17:08.259299 8648 raft_consensus.cc:2454] T af9f9081dffd439a8685396e3403db89 P 5ac6b50428b346a1b5604b42baf02cee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8a62c956c3e46baabad006ee1791b9d in term 0.
I20260923 23:17:08.259410 8715 leader_election.cc:302] T af9f9081dffd439a8685396e3403db89 P b8a62c956c3e46baabad006ee1791b9d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9a094e6b7a64477fb4cda66abefcbdfd, b8a62c956c3e46baabad006ee1791b9d; no voters:
I20260923 23:17:08.259511 8848 raft_consensus.cc:2781] T af9f9081dffd439a8685396e3403db89 P b8a62c956c3e46baabad006ee1791b9d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 23:17:08.259565 8848 raft_consensus.cc:482] T af9f9081dffd439a8685396e3403db89 P b8a62c956c3e46baabad006ee1791b9d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 23:17:08.259588 8848 raft_consensus.cc:3037] T af9f9081dffd439a8685396e3403db89 P b8a62c956c3e46baabad006ee1791b9d [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:17:08.260172 8848 raft_consensus.cc:504] T af9f9081dffd439a8685396e3403db89 P b8a62c956c3e46baabad006ee1791b9d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8a62c956c3e46baabad006ee1791b9d" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 35361 } } peers { permanent_uuid: "9a094e6b7a64477fb4cda66abefcbdfd" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 46547 } } peers { permanent_uuid: "5ac6b50428b346a1b5604b42baf02cee" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 36765 } }
I20260923 23:17:08.260295 8848 leader_election.cc:288] T af9f9081dffd439a8685396e3403db89 P b8a62c956c3e46baabad006ee1791b9d [CANDIDATE]: Term 1 election: Requested vote from peers 9a094e6b7a64477fb4cda66abefcbdfd (127.24.227.193:46547), 5ac6b50428b346a1b5604b42baf02cee (127.24.227.194:36765)
I20260923 23:17:08.260483 8648 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af9f9081dffd439a8685396e3403db89" candidate_uuid: "b8a62c956c3e46baabad006ee1791b9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ac6b50428b346a1b5604b42baf02cee"
I20260923 23:17:08.260479 8514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af9f9081dffd439a8685396e3403db89" candidate_uuid: "b8a62c956c3e46baabad006ee1791b9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a094e6b7a64477fb4cda66abefcbdfd"
I20260923 23:17:08.260560 8514 raft_consensus.cc:3037] T af9f9081dffd439a8685396e3403db89 P 9a094e6b7a64477fb4cda66abefcbdfd [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:17:08.260572 8648 raft_consensus.cc:3037] T af9f9081dffd439a8685396e3403db89 P 5ac6b50428b346a1b5604b42baf02cee [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:17:08.261145 8514 raft_consensus.cc:2454] T af9f9081dffd439a8685396e3403db89 P 9a094e6b7a64477fb4cda66abefcbdfd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8a62c956c3e46baabad006ee1791b9d in term 1.
I20260923 23:17:08.261154 8648 raft_consensus.cc:2454] T af9f9081dffd439a8685396e3403db89 P 5ac6b50428b346a1b5604b42baf02cee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8a62c956c3e46baabad006ee1791b9d in term 1.
I20260923 23:17:08.261312 8715 leader_election.cc:302] T af9f9081dffd439a8685396e3403db89 P b8a62c956c3e46baabad006ee1791b9d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9a094e6b7a64477fb4cda66abefcbdfd, b8a62c956c3e46baabad006ee1791b9d; no voters:
I20260923 23:17:08.261411 8848 raft_consensus.cc:2781] T af9f9081dffd439a8685396e3403db89 P b8a62c956c3e46baabad006ee1791b9d [term 1 FOLLOWER]: Leader election won for term 1
I20260923 23:17:08.261556 8848 raft_consensus.cc:686] T af9f9081dffd439a8685396e3403db89 P b8a62c956c3e46baabad006ee1791b9d [term 1 LEADER]: Becoming Leader. State: Replica: b8a62c956c3e46baabad006ee1791b9d, State: Running, Role: LEADER
I20260923 23:17:08.261643 8848 consensus_queue.cc:237] T af9f9081dffd439a8685396e3403db89 P b8a62c956c3e46baabad006ee1791b9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8a62c956c3e46baabad006ee1791b9d" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 35361 } } peers { permanent_uuid: "9a094e6b7a64477fb4cda66abefcbdfd" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 46547 } } peers { permanent_uuid: "5ac6b50428b346a1b5604b42baf02cee" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 36765 } }
I20260923 23:17:08.262285 8368 catalog_manager.cc:5495] T af9f9081dffd439a8685396e3403db89 P b8a62c956c3e46baabad006ee1791b9d reported cstate change: term changed from 0 to 1, leader changed from <none> to b8a62c956c3e46baabad006ee1791b9d (127.24.227.195). New cstate: current_term: 1 leader_uuid: "b8a62c956c3e46baabad006ee1791b9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8a62c956c3e46baabad006ee1791b9d" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 35361 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a094e6b7a64477fb4cda66abefcbdfd" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 46547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ac6b50428b346a1b5604b42baf02cee" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 36765 } health_report { overall_health: UNKNOWN } } }
I20260923 23:17:08.269264 8839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:17:08.267279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34078 (local address 127.24.227.195:35361)
0923 23:17:08.267318 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0923 23:17:08.267319 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 23:17:08.267329 (+ 10us) server_negotiation.cc:407] Connection header received
0923 23:17:08.267378 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:17:08.267379 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:17:08.267400 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:17:08.267453 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 23:17:08.267526 (+ 73us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:17:08.268007 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:17:08.268665 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:17:08.268834 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:17:08.268879 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:17:08.268995 (+ 116us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 23:17:08.269076 (+ 81us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 23:17:08.269131 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0923 23:17:08.269148 (+ 17us) server_negotiation.cc:299] Negotiation successful
0923 23:17:08.269161 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260923 23:17:08.270759 8648 raft_consensus.cc:1261] T af9f9081dffd439a8685396e3403db89 P 5ac6b50428b346a1b5604b42baf02cee [term 1 FOLLOWER]: Refusing update from remote peer b8a62c956c3e46baabad006ee1791b9d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 23:17:08.270787 8514 raft_consensus.cc:1261] T af9f9081dffd439a8685396e3403db89 P 9a094e6b7a64477fb4cda66abefcbdfd [term 1 FOLLOWER]: Refusing update from remote peer b8a62c956c3e46baabad006ee1791b9d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 23:17:08.271091 8852 consensus_queue.cc:1035] T af9f9081dffd439a8685396e3403db89 P b8a62c956c3e46baabad006ee1791b9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a094e6b7a64477fb4cda66abefcbdfd" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 46547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 23:17:08.271186 8848 consensus_queue.cc:1035] T af9f9081dffd439a8685396e3403db89 P b8a62c956c3e46baabad006ee1791b9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ac6b50428b346a1b5604b42baf02cee" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 36765 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 23:17:08.273618 8855 mvcc.cc:205] Tried to move back new op lower bound from 7332681434194718720 to 7332681434160267264. Current Snapshot: MvccSnapshot[applied={T|T < 7332681434194718720}]
I20260923 23:17:08.274256 8857 mvcc.cc:205] Tried to move back new op lower bound from 7332681434194718720 to 7332681434160267264. Current Snapshot: MvccSnapshot[applied={T|T < 7332681434194718720}]
I20260923 23:17:08.274322 8856 mvcc.cc:205] Tried to move back new op lower bound from 7332681434194718720 to 7332681434160267264. Current Snapshot: MvccSnapshot[applied={T|T < 7332681434194718720}]
I20260923 23:17:08.275692 8367 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33960:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 23:17:08.275779 8367 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33960:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 23:17:08.276523 8367 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 70625401bbe84b8faa08091c510ffafe: Sending DeleteTablet for 3 replicas of tablet af9f9081dffd439a8685396e3403db89
I20260923 23:17:08.276840 8494 tablet_service.cc:1514] Processing DeleteTablet for tablet af9f9081dffd439a8685396e3403db89 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 23:17:08 UTC) from {username='kudu', principal='kudu/127.24.227.254@KRBTEST.COM'} at 127.0.0.1:35392
I20260923 23:17:08.276963 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 8431
I20260923 23:17:08.277004 8628 tablet_service.cc:1514] Processing DeleteTablet for tablet af9f9081dffd439a8685396e3403db89 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 23:17:08 UTC) from {username='kudu', principal='kudu/127.24.227.254@KRBTEST.COM'} at 127.0.0.1:42714
I20260923 23:17:08.277040 8762 tablet_service.cc:1514] Processing DeleteTablet for tablet af9f9081dffd439a8685396e3403db89 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 23:17:08 UTC) from {username='kudu', principal='kudu/127.24.227.254@KRBTEST.COM'} at 127.0.0.1:34070
I20260923 23:17:08.277302 8867 tablet_replica.cc:330] T af9f9081dffd439a8685396e3403db89 P 5ac6b50428b346a1b5604b42baf02cee: stopping tablet replica
I20260923 23:17:08.277475 8867 raft_consensus.cc:2229] T af9f9081dffd439a8685396e3403db89 P 5ac6b50428b346a1b5604b42baf02cee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 23:17:08.277493 8866 tablet_replica.cc:330] T af9f9081dffd439a8685396e3403db89 P b8a62c956c3e46baabad006ee1791b9d: stopping tablet replica
I20260923 23:17:08.277577 8866 raft_consensus.cc:2229] T af9f9081dffd439a8685396e3403db89 P b8a62c956c3e46baabad006ee1791b9d [term 1 LEADER]: Raft consensus shutting down.
I20260923 23:17:08.277674 8867 raft_consensus.cc:2258] T af9f9081dffd439a8685396e3403db89 P 5ac6b50428b346a1b5604b42baf02cee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 23:17:08.277684 8866 raft_consensus.cc:2258] T af9f9081dffd439a8685396e3403db89 P b8a62c956c3e46baabad006ee1791b9d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 23:17:08.278112 8866 ts_tablet_manager.cc:1905] T af9f9081dffd439a8685396e3403db89 P b8a62c956c3e46baabad006ee1791b9d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 23:17:08.278112 8867 ts_tablet_manager.cc:1905] T af9f9081dffd439a8685396e3403db89 P 5ac6b50428b346a1b5604b42baf02cee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 23:17:08.279744 8867 ts_tablet_manager.cc:1918] T af9f9081dffd439a8685396e3403db89 P 5ac6b50428b346a1b5604b42baf02cee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 23:17:08.279753 8866 ts_tablet_manager.cc:1918] T af9f9081dffd439a8685396e3403db89 P b8a62c956c3e46baabad006ee1791b9d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 23:17:08.279814 8866 log.cc:1196] T af9f9081dffd439a8685396e3403db89 P b8a62c956c3e46baabad006ee1791b9d: Deleting WAL directory at /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-2/wal/wals/af9f9081dffd439a8685396e3403db89
I20260923 23:17:08.279975 8867 log.cc:1196] T af9f9081dffd439a8685396e3403db89 P 5ac6b50428b346a1b5604b42baf02cee: Deleting WAL directory at /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-1/wal/wals/af9f9081dffd439a8685396e3403db89
I20260923 23:17:08.280119 8866 ts_tablet_manager.cc:1939] T af9f9081dffd439a8685396e3403db89 P b8a62c956c3e46baabad006ee1791b9d: Deleting consensus metadata
I20260923 23:17:08.280407 8867 ts_tablet_manager.cc:1939] T af9f9081dffd439a8685396e3403db89 P 5ac6b50428b346a1b5604b42baf02cee: Deleting consensus metadata
I20260923 23:17:08.280805 8356 catalog_manager.cc:4841] TS 5ac6b50428b346a1b5604b42baf02cee (127.24.227.194:36765): tablet af9f9081dffd439a8685396e3403db89 (table test-table [id=52fe3a37f4c440298129b1a8b6726b11]) successfully deleted
I20260923 23:17:08.283982 8355 catalog_manager.cc:4841] TS b8a62c956c3e46baabad006ee1791b9d (127.24.227.195:35361): tablet af9f9081dffd439a8685396e3403db89 (table test-table [id=52fe3a37f4c440298129b1a8b6726b11]) successfully deleted
W20260923 23:17:08.284245 8353 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.227.193:46547 (error 108)
I20260923 23:17:08.284490 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 8565
I20260923 23:17:08.284689 8837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:17:08.284583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.193:46547 (local address 127.0.0.1:35398)
0923 23:17:08.284626 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0923 23:17:08.284656 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.193:46547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260923 23:17:08.285377 8353 catalog_manager.cc:4584] TS 9a094e6b7a64477fb4cda66abefcbdfd (127.24.227.193:46547): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet af9f9081dffd439a8685396e3403db89: Network error: Client connection negotiation failed: client connection to 127.24.227.193:46547: connect: Connection refused (error 111)
I20260923 23:17:08.289003 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 8699
I20260923 23:17:08.293691 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 8337
I20260923 23:17:28.298836 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.254:40121
--webserver_interface=127.24.227.254
--webserver_port=41419
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab
--principal=kudu/127.24.227.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:33557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.227.254:40121
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:17:28.377166 8869 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:17:28.377341 8869 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:17:28.377362 8869 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:17:28.378477 8869 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260923 23:17:28.378516 8869 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260923 23:17:28.378530 8869 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260923 23:17:28.378566 8869 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 23:17:28.378593 8869 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:17:28.378611 8869 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:17:28.378638 8869 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 23:17:28.378667 8869 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 23:17:28.380195 8869 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:33557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.24.227.254:40121
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab
--principal=kudu/127.24.227.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.254:40121
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.227.254
--webserver_port=41419
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8869
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:17:28.380421 8869 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:17:28.380654 8869 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260923 23:17:28.388983 8869 server_base.cc:1034] running on GCE node
W20260923 23:17:28.388949 8876 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:17:28.389109 8878 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:17:28.388944 8875 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:17:28.389798 8869 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:17:28.390093 8869 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:17:28.391270 8869 hybrid_clock.cc:648] HybridClock initialized: now 1790205448391264 us; error 46 us; skew 500 ppm
Sep 23 23:17:28 dist-test-slave-754n krb5kdc[8324](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205448, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:17:28.393641 8869 init.cc:377] Logged in from keytab as kudu/127.24.227.254@KRBTEST.COM (short username kudu)
I20260923 23:17:28.394102 8869 webserver.cc:460] Webserver started at http://127.24.227.254:41419/ using document root <none> and password file <none>
I20260923 23:17:28.394318 8869 fs_manager.cc:359] Metadata directory not provided
I20260923 23:17:28.394393 8869 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:17:28.395690 8869 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 23:17:28.396440 8885 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:17:28.396600 8869 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 23:17:28.396695 8869 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/master-0/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/master-0/wal
uuid: "70625401bbe84b8faa08091c510ffafe"
format_stamp: "Formatted at 2026-09-23 23:17:07 on dist-test-slave-754n"
server_key: "0315412c130e13af12293caf8c67a1a2"
server_key_iv: "61d15ddcff79eeabd9a47b62fdc13afd"
server_key_version: "encryptionkey@0"
I20260923 23:17:28.397064 8869 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:17:28.409332 8869 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:17:28.410975 8869 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:17:28.411128 8869 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:17:28.415237 8869 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.254:40121
I20260923 23:17:28.415267 8937 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.254:40121 every 8 connection(s)
I20260923 23:17:28.415633 8869 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
I20260923 23:17:28.416949 8938 sys_catalog.cc:263] Verifying existing consensus state
I20260923 23:17:28.417572 8938 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70625401bbe84b8faa08091c510ffafe: Bootstrap starting.
I20260923 23:17:28.423364 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 8869
I20260923 23:17:28.423705 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.227.193:46547
--local_ip_for_outbound_sockets=127.24.227.193
--tserver_master_addrs=127.24.227.254:40121
--webserver_port=37869
--webserver_interface=127.24.227.193
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.193.keytab
--principal=kudu/127.24.227.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:33557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 23:17:28.426079 8938 log.cc:824] T 00000000000000000000000000000000 P 70625401bbe84b8faa08091c510ffafe: Log is configured to *not* fsync() on all Append() calls
I20260923 23:17:28.427958 8938 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70625401bbe84b8faa08091c510ffafe: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 23:17:28.428293 8938 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70625401bbe84b8faa08091c510ffafe: Bootstrap complete.
I20260923 23:17:28.430406 8938 raft_consensus.cc:348] T 00000000000000000000000000000000 P 70625401bbe84b8faa08091c510ffafe [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70625401bbe84b8faa08091c510ffafe" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 40121 } }
I20260923 23:17:28.430598 8938 raft_consensus.cc:729] T 00000000000000000000000000000000 P 70625401bbe84b8faa08091c510ffafe [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70625401bbe84b8faa08091c510ffafe, State: Initialized, Role: FOLLOWER
I20260923 23:17:28.430737 8938 consensus_queue.cc:260] T 00000000000000000000000000000000 P 70625401bbe84b8faa08091c510ffafe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70625401bbe84b8faa08091c510ffafe" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 40121 } }
I20260923 23:17:28.430791 8938 raft_consensus.cc:388] T 00000000000000000000000000000000 P 70625401bbe84b8faa08091c510ffafe [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 23:17:28.430846 8938 raft_consensus.cc:482] T 00000000000000000000000000000000 P 70625401bbe84b8faa08091c510ffafe [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 23:17:28.430905 8938 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 70625401bbe84b8faa08091c510ffafe [term 1 FOLLOWER]: Advancing to term 2
I20260923 23:17:28.431840 8938 raft_consensus.cc:504] T 00000000000000000000000000000000 P 70625401bbe84b8faa08091c510ffafe [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70625401bbe84b8faa08091c510ffafe" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 40121 } }
I20260923 23:17:28.432001 8938 leader_election.cc:302] T 00000000000000000000000000000000 P 70625401bbe84b8faa08091c510ffafe [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 70625401bbe84b8faa08091c510ffafe; no voters:
I20260923 23:17:28.432180 8938 leader_election.cc:288] T 00000000000000000000000000000000 P 70625401bbe84b8faa08091c510ffafe [CANDIDATE]: Term 2 election: Requested vote from peers
I20260923 23:17:28.432211 8942 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 70625401bbe84b8faa08091c510ffafe [term 2 FOLLOWER]: Leader election won for term 2
I20260923 23:17:28.432317 8942 raft_consensus.cc:686] T 00000000000000000000000000000000 P 70625401bbe84b8faa08091c510ffafe [term 2 LEADER]: Becoming Leader. State: Replica: 70625401bbe84b8faa08091c510ffafe, State: Running, Role: LEADER
I20260923 23:17:28.432396 8942 consensus_queue.cc:237] T 00000000000000000000000000000000 P 70625401bbe84b8faa08091c510ffafe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70625401bbe84b8faa08091c510ffafe" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 40121 } }
I20260923 23:17:28.432422 8938 sys_catalog.cc:564] T 00000000000000000000000000000000 P 70625401bbe84b8faa08091c510ffafe [sys.catalog]: configured and running, proceeding with master startup.
I20260923 23:17:28.432664 8942 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70625401bbe84b8faa08091c510ffafe [sys.catalog]: SysCatalogTable state changed. Reason: New leader 70625401bbe84b8faa08091c510ffafe. Latest consensus state: current_term: 2 leader_uuid: "70625401bbe84b8faa08091c510ffafe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70625401bbe84b8faa08091c510ffafe" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 40121 } } }
I20260923 23:17:28.432734 8942 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70625401bbe84b8faa08091c510ffafe [sys.catalog]: This master's current role is: LEADER
I20260923 23:17:28.432669 8943 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70625401bbe84b8faa08091c510ffafe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "70625401bbe84b8faa08091c510ffafe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70625401bbe84b8faa08091c510ffafe" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 40121 } } }
I20260923 23:17:28.432807 8943 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70625401bbe84b8faa08091c510ffafe [sys.catalog]: This master's current role is: LEADER
I20260923 23:17:28.433023 8946 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 23:17:28.433748 8946 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 23:17:28.433838 8946 catalog_manager.cc:1259] Loaded cluster ID: f235dee3b5b04dd48ee9f905690e8ef1
I20260923 23:17:28.433858 8946 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 23:17:28.435074 8946 catalog_manager.cc:1504] Loading token signing keys...
I20260923 23:17:28.435231 8946 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 70625401bbe84b8faa08091c510ffafe: Loaded TSK: 0
I20260923 23:17:28.449334 8946 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 70625401bbe84b8faa08091c510ffafe: Generated new TSK 1
I20260923 23:17:28.449406 8946 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260923 23:17:28.503403 8940 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:17:28.503613 8940 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:17:28.503647 8940 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:17:28.504835 8940 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:17:28.504904 8940 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:17:28.504966 8940 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.227.193
I20260923 23:17:28.506379 8940 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:33557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.193.keytab
--principal=kudu/127.24.227.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.227.193:46547
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.227.193
--webserver_port=37869
--tserver_master_addrs=127.24.227.254:40121
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8940
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.227.193
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:17:28.506641 8940 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:17:28.506846 8940 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:17:28.512568 8966 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:17:28.512672 8940 server_base.cc:1034] running on GCE node
W20260923 23:17:28.512568 8965 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:17:28.512836 8968 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:17:28.513048 8940 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:17:28.513319 8940 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:17:28.514479 8940 hybrid_clock.cc:648] HybridClock initialized: now 1790205448514467 us; error 30 us; skew 500 ppm
Sep 23 23:17:28 dist-test-slave-754n krb5kdc[8324](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205448, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:17:28.521660 8940 init.cc:377] Logged in from keytab as kudu/127.24.227.193@KRBTEST.COM (short username kudu)
I20260923 23:17:28.522097 8940 webserver.cc:460] Webserver started at http://127.24.227.193:37869/ using document root <none> and password file <none>
I20260923 23:17:28.522318 8940 fs_manager.cc:359] Metadata directory not provided
I20260923 23:17:28.522399 8940 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:17:28.523772 8940 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 23:17:28.524583 8975 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:17:28.524752 8940 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260923 23:17:28.524854 8940 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-0/wal
uuid: "9a094e6b7a64477fb4cda66abefcbdfd"
format_stamp: "Formatted at 2026-09-23 23:17:07 on dist-test-slave-754n"
server_key: "5fb09fd0f1d54d0105773590ccf76e2f"
server_key_iv: "f3b3e0fe422ee48e48d46a1ec2389715"
server_key_version: "encryptionkey@0"
I20260923 23:17:28.525197 8940 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:17:28.531744 8940 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:17:28.534019 8940 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:17:28.534159 8940 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:17:28.534356 8940 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 23:17:28.534866 8982 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260923 23:17:28.536705 8940 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260923 23:17:28.536758 8940 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260923 23:17:28.536795 8940 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260923 23:17:28.537322 8940 ts_tablet_manager.cc:610] Registered 1 tablets
I20260923 23:17:28.537376 8940 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260923 23:17:28.537439 8982 tablet_bootstrap.cc:492] T af9f9081dffd439a8685396e3403db89 P 9a094e6b7a64477fb4cda66abefcbdfd: Bootstrap starting.
I20260923 23:17:28.544320 8940 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.193:46547
I20260923 23:17:28.544632 8940 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-0/data/info.pb
I20260923 23:17:28.547708 8982 log.cc:824] T af9f9081dffd439a8685396e3403db89 P 9a094e6b7a64477fb4cda66abefcbdfd: Log is configured to *not* fsync() on all Append() calls
I20260923 23:17:28.548820 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 8940
I20260923 23:17:28.549115 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.227.194:36765
--local_ip_for_outbound_sockets=127.24.227.194
--tserver_master_addrs=127.24.227.254:40121
--webserver_port=34375
--webserver_interface=127.24.227.194
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.194.keytab
--principal=kudu/127.24.227.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:33557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 23:17:28.550060 8982 tablet_bootstrap.cc:492] T af9f9081dffd439a8685396e3403db89 P 9a094e6b7a64477fb4cda66abefcbdfd: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 23:17:28.550356 8982 tablet_bootstrap.cc:492] T af9f9081dffd439a8685396e3403db89 P 9a094e6b7a64477fb4cda66abefcbdfd: Bootstrap complete.
I20260923 23:17:28.551596 8982 ts_tablet_manager.cc:1397] T af9f9081dffd439a8685396e3403db89 P 9a094e6b7a64477fb4cda66abefcbdfd: Time spent bootstrapping tablet: real 0.014s user 0.001s sys 0.009s
I20260923 23:17:28.552917 8982 raft_consensus.cc:348] T af9f9081dffd439a8685396e3403db89 P 9a094e6b7a64477fb4cda66abefcbdfd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8a62c956c3e46baabad006ee1791b9d" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 35361 } } peers { permanent_uuid: "9a094e6b7a64477fb4cda66abefcbdfd" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 46547 } } peers { permanent_uuid: "5ac6b50428b346a1b5604b42baf02cee" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 36765 } }
I20260923 23:17:28.553207 8982 raft_consensus.cc:729] T af9f9081dffd439a8685396e3403db89 P 9a094e6b7a64477fb4cda66abefcbdfd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a094e6b7a64477fb4cda66abefcbdfd, State: Initialized, Role: FOLLOWER
I20260923 23:17:28.553314 8982 consensus_queue.cc:260] T af9f9081dffd439a8685396e3403db89 P 9a094e6b7a64477fb4cda66abefcbdfd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8a62c956c3e46baabad006ee1791b9d" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 35361 } } peers { permanent_uuid: "9a094e6b7a64477fb4cda66abefcbdfd" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 46547 } } peers { permanent_uuid: "5ac6b50428b346a1b5604b42baf02cee" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 36765 } }
I20260923 23:17:28.553568 8982 ts_tablet_manager.cc:1428] T af9f9081dffd439a8685396e3403db89 P 9a094e6b7a64477fb4cda66abefcbdfd: Time spent starting tablet: real 0.002s user 0.004s sys 0.002s
Sep 23 23:17:28 dist-test-slave-754n krb5kdc[8324](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205448, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.193@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
I20260923 23:17:28.556013 9089 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.193:46547 every 8 connection(s)
I20260923 23:17:28.559445 9092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:17:28.545244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:40121 (local address 127.24.227.193:59133)
0923 23:17:28.545436 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0923 23:17:28.545479 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0923 23:17:28.545873 (+ 394us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:17:28.548418 (+ 2545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:17:28.548422 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:17:28.548785 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:17:28.549121 (+ 336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:17:28.549127 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:17:28.552228 (+ 3101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:17:28.552229 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:17:28.553447 (+ 1218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:17:28.553449 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:17:28.554165 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:17:28.554167 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:17:28.554290 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:17:28.554809 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:17:28.554826 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:17:28.556181 (+ 1355us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:17:28.557871 (+ 1690us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:17:28.557877 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:17:28.557878 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:17:28.558124 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:17:28.558386 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:17:28.558387 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:17:28.558387 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:17:28.558498 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:17:28.558854 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:17:28.558860 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:17:28.558998 (+ 138us) client_negotiation.cc:769] Sending connection context
0923 23:17:28.559096 (+ 98us) client_negotiation.cc:240] Negotiation successful
0923 23:17:28.559271 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":120,"threads_started":1}
I20260923 23:17:28.559468 9093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:17:28.545306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.193:59133 (local address 127.24.227.254:40121)
0923 23:17:28.547986 (+ 2680us) server_negotiation.cc:206] Beginning negotiation
0923 23:17:28.547995 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 23:17:28.548030 (+ 35us) server_negotiation.cc:407] Connection header received
0923 23:17:28.548172 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:17:28.548179 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:17:28.548231 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:17:28.548380 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:17:28.550598 (+ 2218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:17:28.552100 (+ 1502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:17:28.553694 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:17:28.554064 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:17:28.554177 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:17:28.556293 (+ 2116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:17:28.556297 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:17:28.556297 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:17:28.556307 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:17:28.557775 (+ 1468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:17:28.558221 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:17:28.558224 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:17:28.558224 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:17:28.558314 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:17:28.558573 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:17:28.558574 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:17:28.558574 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:17:28.558780 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:17:28.558873 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0923 23:17:28.559160 (+ 287us) server_negotiation.cc:299] Negotiation successful
0923 23:17:28.559286 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2635,"thread_start_us":88,"threads_started":1}
I20260923 23:17:28.560060 9090 heartbeater.cc:347] Connected to a master server at 127.24.227.254:40121
I20260923 23:17:28.560166 9090 heartbeater.cc:464] Registering TS with master...
I20260923 23:17:28.560335 9090 heartbeater.cc:511] Master 127.24.227.254:40121 requested a full tablet report, sending...
I20260923 23:17:28.560957 8902 ts_manager.cc:194] Registered new tserver with Master: 9a094e6b7a64477fb4cda66abefcbdfd (127.24.227.193:46547)
I20260923 23:17:28.561882 8902 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.227.193@KRBTEST.COM'} at 127.24.227.193:59133
I20260923 23:17:28.562615 9090 heartbeater.cc:503] Master 127.24.227.254:40121 was elected leader, sending a full tablet report...
I20260923 23:17:28.566896 9097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:17:28.561456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.193:46547 (local address 127.0.0.1:47442)
0923 23:17:28.564001 (+ 2545us) negotiation.cc:107] Waiting for socket to connect
0923 23:17:28.564028 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0923 23:17:28.564116 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:17:28.564296 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:17:28.564298 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:17:28.564300 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:17:28.564356 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:17:28.564357 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:17:28.565160 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:17:28.565161 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:17:28.565936 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:17:28.565938 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:17:28.566600 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:17:28.566601 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:17:28.566679 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:17:28.566690 (+ 11us) client_negotiation.cc:769] Sending connection context
0923 23:17:28.566779 (+ 89us) client_negotiation.cc:240] Negotiation successful
0923 23:17:28.566832 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2521,"thread_start_us":78,"threads_started":1}
I20260923 23:17:28.566893 9098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:17:28.562153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47442 (local address 127.24.227.193:46547)
0923 23:17:28.562286 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0923 23:17:28.562290 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 23:17:28.564100 (+ 1810us) server_negotiation.cc:407] Connection header received
0923 23:17:28.564202 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:17:28.564204 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:17:28.564218 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:17:28.564289 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:17:28.564421 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:17:28.565059 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:17:28.566057 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:17:28.566515 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:17:28.566601 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:17:28.566651 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0923 23:17:28.566796 (+ 145us) server_negotiation.cc:299] Negotiation successful
0923 23:17:28.566831 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":70,"threads_started":1}
I20260923 23:17:28.568998 9024 tablet_service.cc:1514] Processing DeleteTablet for tablet af9f9081dffd439a8685396e3403db89 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 23:17:08 UTC) from {username='kudu', principal='kudu/127.24.227.254@KRBTEST.COM'} at 127.0.0.1:47442
I20260923 23:17:28.570151 9099 tablet_replica.cc:330] stopping tablet replica
I20260923 23:17:28.570245 9099 raft_consensus.cc:2229] T af9f9081dffd439a8685396e3403db89 P 9a094e6b7a64477fb4cda66abefcbdfd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 23:17:28.570299 9099 raft_consensus.cc:2258] T af9f9081dffd439a8685396e3403db89 P 9a094e6b7a64477fb4cda66abefcbdfd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 23:17:28.570552 9099 ts_tablet_manager.cc:1905] T af9f9081dffd439a8685396e3403db89 P 9a094e6b7a64477fb4cda66abefcbdfd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 23:17:28.573434 9099 ts_tablet_manager.cc:1918] T af9f9081dffd439a8685396e3403db89 P 9a094e6b7a64477fb4cda66abefcbdfd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 23:17:28.573510 9099 log.cc:1196] T af9f9081dffd439a8685396e3403db89 P 9a094e6b7a64477fb4cda66abefcbdfd: Deleting WAL directory at /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-0/wal/wals/af9f9081dffd439a8685396e3403db89
I20260923 23:17:28.573752 9099 ts_tablet_manager.cc:1939] T af9f9081dffd439a8685396e3403db89 P 9a094e6b7a64477fb4cda66abefcbdfd: Deleting consensus metadata
I20260923 23:17:28.574467 8886 catalog_manager.cc:4841] TS 9a094e6b7a64477fb4cda66abefcbdfd (127.24.227.193:46547): tablet af9f9081dffd439a8685396e3403db89 (table test-table [id=52fe3a37f4c440298129b1a8b6726b11]) successfully deleted
W20260923 23:17:28.645043 9095 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:17:28.645222 9095 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:17:28.645244 9095 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:17:28.646359 9095 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:17:28.646397 9095 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:17:28.646425 9095 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.227.194
I20260923 23:17:28.647836 9095 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:33557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.194.keytab
--principal=kudu/127.24.227.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.227.194:36765
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.227.194
--webserver_port=34375
--tserver_master_addrs=127.24.227.254:40121
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9095
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.227.194
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:17:28.648077 9095 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:17:28.648253 9095 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:17:28.652840 9104 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:17:28.652863 9105 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:17:28.652881 9095 server_base.cc:1034] running on GCE node
W20260923 23:17:28.652840 9107 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:17:28.653224 9095 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:17:28.653470 9095 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:17:28.654604 9095 hybrid_clock.cc:648] HybridClock initialized: now 1790205448654596 us; error 27 us; skew 500 ppm
Sep 23 23:17:28 dist-test-slave-754n krb5kdc[8324](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205448, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:17:28.656868 9095 init.cc:377] Logged in from keytab as kudu/127.24.227.194@KRBTEST.COM (short username kudu)
I20260923 23:17:28.657194 9095 webserver.cc:460] Webserver started at http://127.24.227.194:34375/ using document root <none> and password file <none>
I20260923 23:17:28.657363 9095 fs_manager.cc:359] Metadata directory not provided
I20260923 23:17:28.657409 9095 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:17:28.658584 9095 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 23:17:28.659071 9114 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:17:28.659255 9095 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 23:17:28.659329 9095 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-1/wal
uuid: "5ac6b50428b346a1b5604b42baf02cee"
format_stamp: "Formatted at 2026-09-23 23:17:08 on dist-test-slave-754n"
server_key: "2cc6b3f7ca7a86612a83fcbd7ba6a005"
server_key_iv: "463e05b2ac27eb65958963ee9210a15c"
server_key_version: "encryptionkey@0"
I20260923 23:17:28.659580 9095 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:17:28.673324 9095 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:17:28.674798 9095 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:17:28.674893 9095 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:17:28.675055 9095 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 23:17:28.675331 9095 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 23:17:28.675362 9095 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 23:17:28.675415 9095 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 23:17:28.675442 9095 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 23:17:28.681064 9095 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.194:36765
I20260923 23:17:28.681133 9227 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.194:36765 every 8 connection(s)
I20260923 23:17:28.681422 9095 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-1/data/info.pb
I20260923 23:17:28.684973 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 9095
I20260923 23:17:28.685235 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.227.195:35361
--local_ip_for_outbound_sockets=127.24.227.195
--tserver_master_addrs=127.24.227.254:40121
--webserver_port=38179
--webserver_interface=127.24.227.195
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.195.keytab
--principal=kudu/127.24.227.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:33557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 23 23:17:28 dist-test-slave-754n krb5kdc[8324](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205448, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.194@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
I20260923 23:17:28.690429 9093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:17:28.682064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.194:52899 (local address 127.24.227.254:40121)
0923 23:17:28.682123 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0923 23:17:28.682125 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:17:28.682336 (+ 211us) server_negotiation.cc:407] Connection header received
0923 23:17:28.682746 (+ 410us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:17:28.682747 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:17:28.682759 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:17:28.682807 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:17:28.683597 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:17:28.684121 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:17:28.684999 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:17:28.685323 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:17:28.685390 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:17:28.687618 (+ 2228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:17:28.687619 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:17:28.687619 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:17:28.687625 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:17:28.689054 (+ 1429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:17:28.689541 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:17:28.689542 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:17:28.689542 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:17:28.689587 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:17:28.689868 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:17:28.689868 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:17:28.689869 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:17:28.689955 (+ 86us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:17:28.690006 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0923 23:17:28.690314 (+ 308us) server_negotiation.cc:299] Negotiation successful
0923 23:17:28.690373 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260923 23:17:28.690686 9230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:17:28.682061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:40121 (local address 127.24.227.194:52899)
0923 23:17:28.682230 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0923 23:17:28.682261 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0923 23:17:28.682662 (+ 401us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:17:28.682872 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:17:28.682875 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:17:28.683352 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:17:28.683521 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:17:28.683525 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:17:28.684192 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:17:28.684192 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:17:28.684921 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:17:28.684923 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:17:28.685476 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:17:28.685477 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:17:28.685628 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:17:28.686083 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:17:28.686092 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:17:28.687529 (+ 1437us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:17:28.689135 (+ 1606us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:17:28.689142 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:17:28.689144 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:17:28.689460 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:17:28.689650 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:17:28.689651 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:17:28.689652 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:17:28.689803 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:17:28.690030 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:17:28.690035 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:17:28.690187 (+ 152us) client_negotiation.cc:769] Sending connection context
0923 23:17:28.690301 (+ 114us) client_negotiation.cc:240] Negotiation successful
0923 23:17:28.690456 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":83,"threads_started":1}
I20260923 23:17:28.691208 9228 heartbeater.cc:347] Connected to a master server at 127.24.227.254:40121
I20260923 23:17:28.691305 9228 heartbeater.cc:464] Registering TS with master...
I20260923 23:17:28.691457 9228 heartbeater.cc:511] Master 127.24.227.254:40121 requested a full tablet report, sending...
I20260923 23:17:28.691843 8902 ts_manager.cc:194] Registered new tserver with Master: 5ac6b50428b346a1b5604b42baf02cee (127.24.227.194:36765)
I20260923 23:17:28.692234 8902 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.227.194@KRBTEST.COM'} at 127.24.227.194:52899
W20260923 23:17:28.763672 9231 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:17:28.763872 9231 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:17:28.763914 9231 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:17:28.765040 9231 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:17:28.765080 9231 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:17:28.765132 9231 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.227.195
I20260923 23:17:28.766510 9231 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:33557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.195.keytab
--principal=kudu/127.24.227.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.227.195:35361
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.227.195
--webserver_port=38179
--tserver_master_addrs=127.24.227.254:40121
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9231
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.227.195
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:17:28.766721 9231 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:17:28.766929 9231 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:17:28.771493 9237 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:17:28.771523 9236 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:17:28.771494 9239 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:17:28.771727 9231 server_base.cc:1034] running on GCE node
I20260923 23:17:28.771962 9231 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:17:28.772198 9231 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:17:28.773329 9231 hybrid_clock.cc:648] HybridClock initialized: now 1790205448773310 us; error 34 us; skew 500 ppm
Sep 23 23:17:28 dist-test-slave-754n krb5kdc[8324](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205448, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:17:28.775573 9231 init.cc:377] Logged in from keytab as kudu/127.24.227.195@KRBTEST.COM (short username kudu)
I20260923 23:17:28.776080 9231 webserver.cc:460] Webserver started at http://127.24.227.195:38179/ using document root <none> and password file <none>
I20260923 23:17:28.776300 9231 fs_manager.cc:359] Metadata directory not provided
I20260923 23:17:28.776356 9231 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:17:28.777565 9231 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 23:17:28.778081 9246 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:17:28.778229 9231 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 23:17:28.778312 9231 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-2/wal
uuid: "b8a62c956c3e46baabad006ee1791b9d"
format_stamp: "Formatted at 2026-09-23 23:17:08 on dist-test-slave-754n"
server_key: "63c6ad468db4723a9a3d64de89590129"
server_key_iv: "4728729873838112e0e61078612ec59c"
server_key_version: "encryptionkey@0"
I20260923 23:17:28.778592 9231 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:17:28.795329 9231 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:17:28.796993 9231 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:17:28.797099 9231 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:17:28.797238 9231 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 23:17:28.797500 9231 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 23:17:28.797529 9231 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 23:17:28.797547 9231 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 23:17:28.797601 9231 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 23:17:28.803195 9231 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.195:35361
I20260923 23:17:28.803251 9359 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.195:35361 every 8 connection(s)
I20260923 23:17:28.803522 9231 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-2/data/info.pb
Sep 23 23:17:28 dist-test-slave-754n krb5kdc[8324](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205448, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.195@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
I20260923 23:17:28.810160 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 9231
I20260923 23:17:28.811813 9093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:17:28.804200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.195:52273 (local address 127.24.227.254:40121)
0923 23:17:28.804265 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0923 23:17:28.804268 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 23:17:28.804442 (+ 174us) server_negotiation.cc:407] Connection header received
0923 23:17:28.804830 (+ 388us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:17:28.804831 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:17:28.804842 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:17:28.804889 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:17:28.805649 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:17:28.806078 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:17:28.806898 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:17:28.807094 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:17:28.807146 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:17:28.809112 (+ 1966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:17:28.809113 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:17:28.809114 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:17:28.809120 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:17:28.810460 (+ 1340us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:17:28.810932 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:17:28.810934 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:17:28.810934 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:17:28.811001 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:17:28.811225 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:17:28.811226 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:17:28.811227 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:17:28.811374 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:17:28.811446 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0923 23:17:28.811688 (+ 242us) server_negotiation.cc:299] Negotiation successful
0923 23:17:28.811751 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260923 23:17:28.812063 9362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:17:28.804195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:40121 (local address 127.24.227.195:52273)
0923 23:17:28.804330 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0923 23:17:28.804360 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0923 23:17:28.804753 (+ 393us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:17:28.804943 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:17:28.804946 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:17:28.805418 (+ 472us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:17:28.805578 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:17:28.805582 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:17:28.806146 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:17:28.806147 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:17:28.806830 (+ 683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:17:28.806831 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:17:28.807164 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:17:28.807165 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:17:28.807257 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:17:28.807734 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:17:28.807741 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:17:28.809033 (+ 1292us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:17:28.810553 (+ 1520us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:17:28.810559 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:17:28.810561 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:17:28.810849 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:17:28.811068 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:17:28.811068 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:17:28.811068 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:17:28.811160 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:17:28.811441 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:17:28.811445 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:17:28.811543 (+ 98us) client_negotiation.cc:769] Sending connection context
0923 23:17:28.811661 (+ 118us) client_negotiation.cc:240] Negotiation successful
0923 23:17:28.811811 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":78,"threads_started":1}
I20260923 23:17:28.812546 9360 heartbeater.cc:347] Connected to a master server at 127.24.227.254:40121
I20260923 23:17:28.812633 9360 heartbeater.cc:464] Registering TS with master...
I20260923 23:17:28.812786 9360 heartbeater.cc:511] Master 127.24.227.254:40121 requested a full tablet report, sending...
I20260923 23:17:28.813164 8902 ts_manager.cc:194] Registered new tserver with Master: b8a62c956c3e46baabad006ee1791b9d (127.24.227.195:35361)
I20260923 23:17:28.813552 8902 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.227.195@KRBTEST.COM'} at 127.24.227.195:52273
I20260923 23:17:28.816577 9363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:17:28.810538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58168 (local address 127.24.227.254:40121)
0923 23:17:28.810663 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0923 23:17:28.810665 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:17:28.810741 (+ 76us) server_negotiation.cc:407] Connection header received
0923 23:17:28.810792 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:17:28.810792 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:17:28.810803 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:17:28.810886 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:17:28.811614 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:17:28.812330 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:17:28.813324 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:17:28.813507 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:17:28.813556 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:17:28.814636 (+ 1080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:17:28.814637 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:17:28.814637 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:17:28.814643 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:17:28.815772 (+ 1129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:17:28.816094 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:17:28.816095 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:17:28.816095 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:17:28.816142 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:17:28.816274 (+ 132us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:17:28.816274 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:17:28.816274 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:17:28.816347 (+ 73us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:17:28.816453 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0923 23:17:28.816476 (+ 23us) server_negotiation.cc:299] Negotiation successful
0923 23:17:28.816527 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":64,"threads_started":1}
I20260923 23:17:28.817225 25487 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 23:17:28.819423 9363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:17:28.817484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58178 (local address 127.24.227.254:40121)
0923 23:17:28.817533 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0923 23:17:28.817534 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 23:17:28.817600 (+ 66us) server_negotiation.cc:407] Connection header received
0923 23:17:28.817670 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:17:28.817671 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:17:28.817693 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:17:28.817740 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 23:17:28.817799 (+ 59us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:17:28.818252 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:17:28.818882 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:17:28.819056 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:17:28.819102 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:17:28.819211 (+ 109us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 23:17:28.819294 (+ 83us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 23:17:28.819344 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0923 23:17:28.819373 (+ 29us) server_negotiation.cc:299] Negotiation successful
0923 23:17:28.819386 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260923 23:17:28.819680 8902 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:58178:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 23:17:28.821234 8902 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 23:17:28.823022 9024 tablet_service.cc:1467] Processing CreateTablet for tablet d45b67fb4faf4893acadd2a9cfd390f8 (DEFAULT_TABLE table=test-table [id=0ee292dbb36049759cb3bc7c27163a5a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 23:17:28.823194 9024 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d45b67fb4faf4893acadd2a9cfd390f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:17:28.824678 8982 tablet_bootstrap.cc:492] T d45b67fb4faf4893acadd2a9cfd390f8 P 9a094e6b7a64477fb4cda66abefcbdfd: Bootstrap starting.
I20260923 23:17:28.824976 8982 tablet_bootstrap.cc:654] T d45b67fb4faf4893acadd2a9cfd390f8 P 9a094e6b7a64477fb4cda66abefcbdfd: Neither blocks nor log segments found. Creating new log.
I20260923 23:17:28.825487 8982 tablet_bootstrap.cc:492] T d45b67fb4faf4893acadd2a9cfd390f8 P 9a094e6b7a64477fb4cda66abefcbdfd: No bootstrap required, opened a new log
I20260923 23:17:28.825536 8982 ts_tablet_manager.cc:1397] T d45b67fb4faf4893acadd2a9cfd390f8 P 9a094e6b7a64477fb4cda66abefcbdfd: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260923 23:17:28.825639 8982 raft_consensus.cc:348] T d45b67fb4faf4893acadd2a9cfd390f8 P 9a094e6b7a64477fb4cda66abefcbdfd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ac6b50428b346a1b5604b42baf02cee" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 36765 } } peers { permanent_uuid: "b8a62c956c3e46baabad006ee1791b9d" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 35361 } } peers { permanent_uuid: "9a094e6b7a64477fb4cda66abefcbdfd" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 46547 } }
I20260923 23:17:28.825707 8982 raft_consensus.cc:374] T d45b67fb4faf4893acadd2a9cfd390f8 P 9a094e6b7a64477fb4cda66abefcbdfd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:17:28.825752 8982 raft_consensus.cc:729] T d45b67fb4faf4893acadd2a9cfd390f8 P 9a094e6b7a64477fb4cda66abefcbdfd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a094e6b7a64477fb4cda66abefcbdfd, State: Initialized, Role: FOLLOWER
I20260923 23:17:28.825793 8982 consensus_queue.cc:260] T d45b67fb4faf4893acadd2a9cfd390f8 P 9a094e6b7a64477fb4cda66abefcbdfd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ac6b50428b346a1b5604b42baf02cee" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 36765 } } peers { permanent_uuid: "b8a62c956c3e46baabad006ee1791b9d" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 35361 } } peers { permanent_uuid: "9a094e6b7a64477fb4cda66abefcbdfd" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 46547 } }
I20260923 23:17:28.825877 8982 ts_tablet_manager.cc:1428] T d45b67fb4faf4893acadd2a9cfd390f8 P 9a094e6b7a64477fb4cda66abefcbdfd: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 23:17:28.826260 9097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:17:28.822782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.194:36765 (local address 127.0.0.1:45838)
0923 23:17:28.822872 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0923 23:17:28.822882 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 23:17:28.822920 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:17:28.823411 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:17:28.823412 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:17:28.823413 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:17:28.823449 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:17:28.823450 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:17:28.824355 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:17:28.824356 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:17:28.825251 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:17:28.825252 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:17:28.826056 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:17:28.826057 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:17:28.826114 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:17:28.826117 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 23:17:28.826170 (+ 53us) client_negotiation.cc:240] Negotiation successful
0923 23:17:28.826207 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74}
I20260923 23:17:28.826290 9367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:17:28.822880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45838 (local address 127.24.227.194:36765)
0923 23:17:28.823049 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0923 23:17:28.823223 (+ 174us) server_negotiation.cc:399] Waiting for connection header
0923 23:17:28.823235 (+ 12us) server_negotiation.cc:407] Connection header received
0923 23:17:28.823292 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:17:28.823294 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:17:28.823309 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:17:28.823403 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:17:28.823495 (+ 92us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:17:28.824278 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:17:28.825329 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:17:28.825955 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:17:28.826055 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:17:28.826105 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0923 23:17:28.826181 (+ 76us) server_negotiation.cc:299] Negotiation successful
0923 23:17:28.826215 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":98,"threads_started":1}
I20260923 23:17:28.826957 9162 tablet_service.cc:1467] Processing CreateTablet for tablet d45b67fb4faf4893acadd2a9cfd390f8 (DEFAULT_TABLE table=test-table [id=0ee292dbb36049759cb3bc7c27163a5a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 23:17:28.827271 9162 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d45b67fb4faf4893acadd2a9cfd390f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:17:28.828130 9366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:17:28.822852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.195:35361 (local address 127.0.0.1:49756)
0923 23:17:28.822986 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0923 23:17:28.822995 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 23:17:28.823032 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:17:28.823490 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:17:28.823491 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:17:28.823493 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:17:28.823528 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:17:28.823529 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:17:28.824627 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:17:28.824628 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:17:28.825564 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:17:28.825567 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:17:28.826401 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:17:28.826404 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:17:28.826512 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:17:28.826516 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 23:17:28.828002 (+ 1486us) client_negotiation.cc:240] Negotiation successful
0923 23:17:28.828065 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":68,"threads_started":1}
I20260923 23:17:28.828421 9368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:17:28.823089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49756 (local address 127.24.227.195:35361)
0923 23:17:28.823285 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0923 23:17:28.823289 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 23:17:28.823298 (+ 9us) server_negotiation.cc:407] Connection header received
0923 23:17:28.823327 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:17:28.823329 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:17:28.823342 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:17:28.823415 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:17:28.823590 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:17:28.824546 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:17:28.825653 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:17:28.826269 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:17:28.826327 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:17:28.826373 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0923 23:17:28.828311 (+ 1938us) server_negotiation.cc:299] Negotiation successful
0923 23:17:28.828352 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":97,"threads_started":1}
I20260923 23:17:28.828958 9294 tablet_service.cc:1467] Processing CreateTablet for tablet d45b67fb4faf4893acadd2a9cfd390f8 (DEFAULT_TABLE table=test-table [id=0ee292dbb36049759cb3bc7c27163a5a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 23:17:28.829285 9294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d45b67fb4faf4893acadd2a9cfd390f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:17:28.830287 9370 tablet_bootstrap.cc:492] T d45b67fb4faf4893acadd2a9cfd390f8 P 5ac6b50428b346a1b5604b42baf02cee: Bootstrap starting.
I20260923 23:17:28.830835 9370 tablet_bootstrap.cc:654] T d45b67fb4faf4893acadd2a9cfd390f8 P 5ac6b50428b346a1b5604b42baf02cee: Neither blocks nor log segments found. Creating new log.
I20260923 23:17:28.831357 9370 log.cc:824] T d45b67fb4faf4893acadd2a9cfd390f8 P 5ac6b50428b346a1b5604b42baf02cee: Log is configured to *not* fsync() on all Append() calls
I20260923 23:17:28.832019 9372 tablet_bootstrap.cc:492] T d45b67fb4faf4893acadd2a9cfd390f8 P b8a62c956c3e46baabad006ee1791b9d: Bootstrap starting.
I20260923 23:17:28.832167 9370 tablet_bootstrap.cc:492] T d45b67fb4faf4893acadd2a9cfd390f8 P 5ac6b50428b346a1b5604b42baf02cee: No bootstrap required, opened a new log
I20260923 23:17:28.832235 9370 ts_tablet_manager.cc:1397] T d45b67fb4faf4893acadd2a9cfd390f8 P 5ac6b50428b346a1b5604b42baf02cee: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 23:17:28.832649 9372 tablet_bootstrap.cc:654] T d45b67fb4faf4893acadd2a9cfd390f8 P b8a62c956c3e46baabad006ee1791b9d: Neither blocks nor log segments found. Creating new log.
I20260923 23:17:28.833138 9372 log.cc:824] T d45b67fb4faf4893acadd2a9cfd390f8 P b8a62c956c3e46baabad006ee1791b9d: Log is configured to *not* fsync() on all Append() calls
I20260923 23:17:28.833810 9372 tablet_bootstrap.cc:492] T d45b67fb4faf4893acadd2a9cfd390f8 P b8a62c956c3e46baabad006ee1791b9d: No bootstrap required, opened a new log
I20260923 23:17:28.833900 9372 ts_tablet_manager.cc:1397] T d45b67fb4faf4893acadd2a9cfd390f8 P b8a62c956c3e46baabad006ee1791b9d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 23:17:28.834036 9370 raft_consensus.cc:348] T d45b67fb4faf4893acadd2a9cfd390f8 P 5ac6b50428b346a1b5604b42baf02cee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ac6b50428b346a1b5604b42baf02cee" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 36765 } } peers { permanent_uuid: "b8a62c956c3e46baabad006ee1791b9d" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 35361 } } peers { permanent_uuid: "9a094e6b7a64477fb4cda66abefcbdfd" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 46547 } }
I20260923 23:17:28.834215 9370 raft_consensus.cc:374] T d45b67fb4faf4893acadd2a9cfd390f8 P 5ac6b50428b346a1b5604b42baf02cee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:17:28.834236 9370 raft_consensus.cc:729] T d45b67fb4faf4893acadd2a9cfd390f8 P 5ac6b50428b346a1b5604b42baf02cee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ac6b50428b346a1b5604b42baf02cee, State: Initialized, Role: FOLLOWER
I20260923 23:17:28.834388 9370 consensus_queue.cc:260] T d45b67fb4faf4893acadd2a9cfd390f8 P 5ac6b50428b346a1b5604b42baf02cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ac6b50428b346a1b5604b42baf02cee" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 36765 } } peers { permanent_uuid: "b8a62c956c3e46baabad006ee1791b9d" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 35361 } } peers { permanent_uuid: "9a094e6b7a64477fb4cda66abefcbdfd" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 46547 } }
I20260923 23:17:28.834661 9370 ts_tablet_manager.cc:1428] T d45b67fb4faf4893acadd2a9cfd390f8 P 5ac6b50428b346a1b5604b42baf02cee: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260923 23:17:28.834683 9228 heartbeater.cc:503] Master 127.24.227.254:40121 was elected leader, sending a full tablet report...
I20260923 23:17:28.835724 9372 raft_consensus.cc:348] T d45b67fb4faf4893acadd2a9cfd390f8 P b8a62c956c3e46baabad006ee1791b9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ac6b50428b346a1b5604b42baf02cee" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 36765 } } peers { permanent_uuid: "b8a62c956c3e46baabad006ee1791b9d" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 35361 } } peers { permanent_uuid: "9a094e6b7a64477fb4cda66abefcbdfd" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 46547 } }
I20260923 23:17:28.835945 9372 raft_consensus.cc:374] T d45b67fb4faf4893acadd2a9cfd390f8 P b8a62c956c3e46baabad006ee1791b9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:17:28.835989 9372 raft_consensus.cc:729] T d45b67fb4faf4893acadd2a9cfd390f8 P b8a62c956c3e46baabad006ee1791b9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8a62c956c3e46baabad006ee1791b9d, State: Initialized, Role: FOLLOWER
I20260923 23:17:28.836089 9372 consensus_queue.cc:260] T d45b67fb4faf4893acadd2a9cfd390f8 P b8a62c956c3e46baabad006ee1791b9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ac6b50428b346a1b5604b42baf02cee" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 36765 } } peers { permanent_uuid: "b8a62c956c3e46baabad006ee1791b9d" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 35361 } } peers { permanent_uuid: "9a094e6b7a64477fb4cda66abefcbdfd" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 46547 } }
I20260923 23:17:28.836326 9372 ts_tablet_manager.cc:1428] T d45b67fb4faf4893acadd2a9cfd390f8 P b8a62c956c3e46baabad006ee1791b9d: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260923 23:17:28.836345 9360 heartbeater.cc:503] Master 127.24.227.254:40121 was elected leader, sending a full tablet report...
I20260923 23:17:28.913552 9375 raft_consensus.cc:482] T d45b67fb4faf4893acadd2a9cfd390f8 P b8a62c956c3e46baabad006ee1791b9d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 23:17:28.913691 9375 raft_consensus.cc:504] T d45b67fb4faf4893acadd2a9cfd390f8 P b8a62c956c3e46baabad006ee1791b9d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ac6b50428b346a1b5604b42baf02cee" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 36765 } } peers { permanent_uuid: "b8a62c956c3e46baabad006ee1791b9d" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 35361 } } peers { permanent_uuid: "9a094e6b7a64477fb4cda66abefcbdfd" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 46547 } }
I20260923 23:17:28.914067 9375 leader_election.cc:288] T d45b67fb4faf4893acadd2a9cfd390f8 P b8a62c956c3e46baabad006ee1791b9d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ac6b50428b346a1b5604b42baf02cee (127.24.227.194:36765), 9a094e6b7a64477fb4cda66abefcbdfd (127.24.227.193:46547)
I20260923 23:17:28.917027 9367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:17:28.914317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.195:32831 (local address 127.24.227.194:36765)
0923 23:17:28.914361 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0923 23:17:28.914363 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:17:28.914372 (+ 9us) server_negotiation.cc:407] Connection header received
0923 23:17:28.914399 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:17:28.914400 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:17:28.914410 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:17:28.914482 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:17:28.914687 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:17:28.915324 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:17:28.916327 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:17:28.916683 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:17:28.916735 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:17:28.916741 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0923 23:17:28.916910 (+ 169us) server_negotiation.cc:299] Negotiation successful
0923 23:17:28.916954 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260923 23:17:28.917032 9362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:17:28.914220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.194:36765 (local address 127.24.227.195:32831)
0923 23:17:28.914273 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0923 23:17:28.914286 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 23:17:28.914333 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:17:28.914515 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:17:28.914517 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:17:28.914519 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:17:28.914593 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:17:28.914596 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:17:28.915395 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:17:28.915396 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:17:28.916230 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:17:28.916232 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:17:28.916756 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:17:28.916757 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:17:28.916840 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:17:28.916843 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 23:17:28.916884 (+ 41us) client_negotiation.cc:240] Negotiation successful
0923 23:17:28.916929 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260923 23:17:28.917073 9098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:17:28.914355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.195:33595 (local address 127.24.227.193:46547)
0923 23:17:28.914397 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0923 23:17:28.914400 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 23:17:28.914510 (+ 110us) server_negotiation.cc:407] Connection header received
0923 23:17:28.914554 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:17:28.914555 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:17:28.914566 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:17:28.914622 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:17:28.914806 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:17:28.915410 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:17:28.916336 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:17:28.916699 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:17:28.916783 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:17:28.916790 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0923 23:17:28.916958 (+ 168us) server_negotiation.cc:299] Negotiation successful
0923 23:17:28.917005 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 23:17:28.917302 9376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:17:28.914260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.193:46547 (local address 127.24.227.195:33595)
0923 23:17:28.914439 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0923 23:17:28.914448 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 23:17:28.914497 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:17:28.914656 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:17:28.914657 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:17:28.914659 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:17:28.914728 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:17:28.914730 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:17:28.915496 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:17:28.915497 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:17:28.916231 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:17:28.916232 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:17:28.916799 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:17:28.916800 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:17:28.916858 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:17:28.916861 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 23:17:28.916933 (+ 72us) client_negotiation.cc:240] Negotiation successful
0923 23:17:28.916972 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":120,"threads_started":1}
I20260923 23:17:28.917395 9182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d45b67fb4faf4893acadd2a9cfd390f8" candidate_uuid: "b8a62c956c3e46baabad006ee1791b9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ac6b50428b346a1b5604b42baf02cee" is_pre_election: true
I20260923 23:17:28.917550 9182 raft_consensus.cc:2454] T d45b67fb4faf4893acadd2a9cfd390f8 P 5ac6b50428b346a1b5604b42baf02cee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8a62c956c3e46baabad006ee1791b9d in term 0.
I20260923 23:17:28.917760 9250 leader_election.cc:302] T d45b67fb4faf4893acadd2a9cfd390f8 P b8a62c956c3e46baabad006ee1791b9d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ac6b50428b346a1b5604b42baf02cee, b8a62c956c3e46baabad006ee1791b9d; no voters:
I20260923 23:17:28.917877 9375 raft_consensus.cc:2781] T d45b67fb4faf4893acadd2a9cfd390f8 P b8a62c956c3e46baabad006ee1791b9d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 23:17:28.917835 9040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d45b67fb4faf4893acadd2a9cfd390f8" candidate_uuid: "b8a62c956c3e46baabad006ee1791b9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a094e6b7a64477fb4cda66abefcbdfd" is_pre_election: true
I20260923 23:17:28.917925 9375 raft_consensus.cc:482] T d45b67fb4faf4893acadd2a9cfd390f8 P b8a62c956c3e46baabad006ee1791b9d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 23:17:28.917961 9375 raft_consensus.cc:3037] T d45b67fb4faf4893acadd2a9cfd390f8 P b8a62c956c3e46baabad006ee1791b9d [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:17:28.917976 9040 raft_consensus.cc:2454] T d45b67fb4faf4893acadd2a9cfd390f8 P 9a094e6b7a64477fb4cda66abefcbdfd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8a62c956c3e46baabad006ee1791b9d in term 0.
I20260923 23:17:28.918666 9375 raft_consensus.cc:504] T d45b67fb4faf4893acadd2a9cfd390f8 P b8a62c956c3e46baabad006ee1791b9d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ac6b50428b346a1b5604b42baf02cee" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 36765 } } peers { permanent_uuid: "b8a62c956c3e46baabad006ee1791b9d" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 35361 } } peers { permanent_uuid: "9a094e6b7a64477fb4cda66abefcbdfd" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 46547 } }
I20260923 23:17:28.918795 9375 leader_election.cc:288] T d45b67fb4faf4893acadd2a9cfd390f8 P b8a62c956c3e46baabad006ee1791b9d [CANDIDATE]: Term 1 election: Requested vote from peers 5ac6b50428b346a1b5604b42baf02cee (127.24.227.194:36765), 9a094e6b7a64477fb4cda66abefcbdfd (127.24.227.193:46547)
I20260923 23:17:28.918918 9182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d45b67fb4faf4893acadd2a9cfd390f8" candidate_uuid: "b8a62c956c3e46baabad006ee1791b9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ac6b50428b346a1b5604b42baf02cee"
I20260923 23:17:28.918943 9040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d45b67fb4faf4893acadd2a9cfd390f8" candidate_uuid: "b8a62c956c3e46baabad006ee1791b9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a094e6b7a64477fb4cda66abefcbdfd"
I20260923 23:17:28.918979 9182 raft_consensus.cc:3037] T d45b67fb4faf4893acadd2a9cfd390f8 P 5ac6b50428b346a1b5604b42baf02cee [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:17:28.919028 9040 raft_consensus.cc:3037] T d45b67fb4faf4893acadd2a9cfd390f8 P 9a094e6b7a64477fb4cda66abefcbdfd [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:17:28.919661 9182 raft_consensus.cc:2454] T d45b67fb4faf4893acadd2a9cfd390f8 P 5ac6b50428b346a1b5604b42baf02cee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8a62c956c3e46baabad006ee1791b9d in term 1.
I20260923 23:17:28.919808 9250 leader_election.cc:302] T d45b67fb4faf4893acadd2a9cfd390f8 P b8a62c956c3e46baabad006ee1791b9d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ac6b50428b346a1b5604b42baf02cee, b8a62c956c3e46baabad006ee1791b9d; no voters:
I20260923 23:17:28.919833 9040 raft_consensus.cc:2454] T d45b67fb4faf4893acadd2a9cfd390f8 P 9a094e6b7a64477fb4cda66abefcbdfd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8a62c956c3e46baabad006ee1791b9d in term 1.
I20260923 23:17:28.919917 9375 raft_consensus.cc:2781] T d45b67fb4faf4893acadd2a9cfd390f8 P b8a62c956c3e46baabad006ee1791b9d [term 1 FOLLOWER]: Leader election won for term 1
I20260923 23:17:28.920064 9375 raft_consensus.cc:686] T d45b67fb4faf4893acadd2a9cfd390f8 P b8a62c956c3e46baabad006ee1791b9d [term 1 LEADER]: Becoming Leader. State: Replica: b8a62c956c3e46baabad006ee1791b9d, State: Running, Role: LEADER
I20260923 23:17:28.920171 9375 consensus_queue.cc:237] T d45b67fb4faf4893acadd2a9cfd390f8 P b8a62c956c3e46baabad006ee1791b9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ac6b50428b346a1b5604b42baf02cee" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 36765 } } peers { permanent_uuid: "b8a62c956c3e46baabad006ee1791b9d" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 35361 } } peers { permanent_uuid: "9a094e6b7a64477fb4cda66abefcbdfd" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 46547 } }
I20260923 23:17:28.920872 8901 catalog_manager.cc:5495] T d45b67fb4faf4893acadd2a9cfd390f8 P b8a62c956c3e46baabad006ee1791b9d reported cstate change: term changed from 0 to 1, leader changed from <none> to b8a62c956c3e46baabad006ee1791b9d (127.24.227.195). New cstate: current_term: 1 leader_uuid: "b8a62c956c3e46baabad006ee1791b9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ac6b50428b346a1b5604b42baf02cee" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 36765 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b8a62c956c3e46baabad006ee1791b9d" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 35361 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a094e6b7a64477fb4cda66abefcbdfd" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 46547 } health_report { overall_health: UNKNOWN } } }
W20260923 23:17:28.931986 9229 tablet.cc:2369] T d45b67fb4faf4893acadd2a9cfd390f8 P 5ac6b50428b346a1b5604b42baf02cee: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 23:17:28.941809 9368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:17:28.939703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49762 (local address 127.24.227.195:35361)
0923 23:17:28.939752 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0923 23:17:28.939754 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:17:28.939846 (+ 92us) server_negotiation.cc:407] Connection header received
0923 23:17:28.939968 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:17:28.939968 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:17:28.939982 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:17:28.940039 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 23:17:28.940143 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:17:28.940560 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:17:28.941199 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:17:28.941373 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:17:28.941419 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:17:28.941545 (+ 126us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 23:17:28.941629 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 23:17:28.941685 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0923 23:17:28.941721 (+ 36us) server_negotiation.cc:299] Negotiation successful
0923 23:17:28.941735 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260923 23:17:28.943406 9182 raft_consensus.cc:1261] T d45b67fb4faf4893acadd2a9cfd390f8 P 5ac6b50428b346a1b5604b42baf02cee [term 1 FOLLOWER]: Refusing update from remote peer b8a62c956c3e46baabad006ee1791b9d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 23:17:28.943427 9040 raft_consensus.cc:1261] T d45b67fb4faf4893acadd2a9cfd390f8 P 9a094e6b7a64477fb4cda66abefcbdfd [term 1 FOLLOWER]: Refusing update from remote peer b8a62c956c3e46baabad006ee1791b9d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 23:17:28.943709 9377 consensus_queue.cc:1035] T d45b67fb4faf4893acadd2a9cfd390f8 P b8a62c956c3e46baabad006ee1791b9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ac6b50428b346a1b5604b42baf02cee" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 36765 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 23:17:28.943837 9375 consensus_queue.cc:1035] T d45b67fb4faf4893acadd2a9cfd390f8 P b8a62c956c3e46baabad006ee1791b9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a094e6b7a64477fb4cda66abefcbdfd" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 46547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 23:17:28.946275 9380 mvcc.cc:205] Tried to move back new op lower bound from 7332681518869852160 to 7332681518777573376. Current Snapshot: MvccSnapshot[applied={T|T < 7332681518869852160}]
I20260923 23:17:28.946668 9381 mvcc.cc:205] Tried to move back new op lower bound from 7332681518869852160 to 7332681518777573376. Current Snapshot: MvccSnapshot[applied={T|T < 7332681518869852160}]
I20260923 23:17:28.947736 8901 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:58178:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 23:17:28.947829 8901 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:58178:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 23:17:28.948570 8901 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 70625401bbe84b8faa08091c510ffafe: Sending DeleteTablet for 3 replicas of tablet d45b67fb4faf4893acadd2a9cfd390f8
I20260923 23:17:28.948848 9024 tablet_service.cc:1514] Processing DeleteTablet for tablet d45b67fb4faf4893acadd2a9cfd390f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 23:17:28 UTC) from {username='kudu', principal='kudu/127.24.227.254@KRBTEST.COM'} at 127.0.0.1:47442
I20260923 23:17:28.948915 9294 tablet_service.cc:1514] Processing DeleteTablet for tablet d45b67fb4faf4893acadd2a9cfd390f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 23:17:28 UTC) from {username='kudu', principal='kudu/127.24.227.254@KRBTEST.COM'} at 127.0.0.1:49756
I20260923 23:17:28.948957 9099 tablet_replica.cc:330] T d45b67fb4faf4893acadd2a9cfd390f8 P 9a094e6b7a64477fb4cda66abefcbdfd: stopping tablet replica
I20260923 23:17:28.949047 9099 raft_consensus.cc:2229] T d45b67fb4faf4893acadd2a9cfd390f8 P 9a094e6b7a64477fb4cda66abefcbdfd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 23:17:28.949123 9099 raft_consensus.cc:2258] T d45b67fb4faf4893acadd2a9cfd390f8 P 9a094e6b7a64477fb4cda66abefcbdfd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 23:17:28.949160 9162 tablet_service.cc:1514] Processing DeleteTablet for tablet d45b67fb4faf4893acadd2a9cfd390f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 23:17:28 UTC) from {username='kudu', principal='kudu/127.24.227.254@KRBTEST.COM'} at 127.0.0.1:45838
I20260923 23:17:28.949419 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 8940
I20260923 23:17:28.949501 9389 tablet_replica.cc:330] T d45b67fb4faf4893acadd2a9cfd390f8 P 5ac6b50428b346a1b5604b42baf02cee: stopping tablet replica
I20260923 23:17:28.949568 9099 ts_tablet_manager.cc:1905] T d45b67fb4faf4893acadd2a9cfd390f8 P 9a094e6b7a64477fb4cda66abefcbdfd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 23:17:28.950153 9389 raft_consensus.cc:2229] T d45b67fb4faf4893acadd2a9cfd390f8 P 5ac6b50428b346a1b5604b42baf02cee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 23:17:28.950278 9389 raft_consensus.cc:2258] T d45b67fb4faf4893acadd2a9cfd390f8 P 5ac6b50428b346a1b5604b42baf02cee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 23:17:28.950686 9389 ts_tablet_manager.cc:1905] T d45b67fb4faf4893acadd2a9cfd390f8 P 5ac6b50428b346a1b5604b42baf02cee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 23:17:28.952646 9389 ts_tablet_manager.cc:1918] T d45b67fb4faf4893acadd2a9cfd390f8 P 5ac6b50428b346a1b5604b42baf02cee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 23:17:28.952713 9389 log.cc:1196] T d45b67fb4faf4893acadd2a9cfd390f8 P 5ac6b50428b346a1b5604b42baf02cee: Deleting WAL directory at /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-1/wal/wals/d45b67fb4faf4893acadd2a9cfd390f8
I20260923 23:17:28.952946 9389 ts_tablet_manager.cc:1939] T d45b67fb4faf4893acadd2a9cfd390f8 P 5ac6b50428b346a1b5604b42baf02cee: Deleting consensus metadata
I20260923 23:17:28.953372 8889 catalog_manager.cc:4841] TS 5ac6b50428b346a1b5604b42baf02cee (127.24.227.194:36765): tablet d45b67fb4faf4893acadd2a9cfd390f8 (table test-table [id=0ee292dbb36049759cb3bc7c27163a5a]) successfully deleted
I20260923 23:17:28.955986 9388 tablet_replica.cc:330] T d45b67fb4faf4893acadd2a9cfd390f8 P b8a62c956c3e46baabad006ee1791b9d: stopping tablet replica
I20260923 23:17:28.956120 9388 raft_consensus.cc:2229] T d45b67fb4faf4893acadd2a9cfd390f8 P b8a62c956c3e46baabad006ee1791b9d [term 1 LEADER]: Raft consensus shutting down.
I20260923 23:17:28.956239 9388 raft_consensus.cc:2258] T d45b67fb4faf4893acadd2a9cfd390f8 P b8a62c956c3e46baabad006ee1791b9d [term 1 FOLLOWER]: Raft consensus is shut down!
W20260923 23:17:28.956637 8886 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.227.193:46547 (error 108)
I20260923 23:17:28.956686 9388 ts_tablet_manager.cc:1905] T d45b67fb4faf4893acadd2a9cfd390f8 P b8a62c956c3e46baabad006ee1791b9d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 23:17:28.956959 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 9095
I20260923 23:17:28.959139 9388 ts_tablet_manager.cc:1918] T d45b67fb4faf4893acadd2a9cfd390f8 P b8a62c956c3e46baabad006ee1791b9d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 23:17:28.959213 9388 log.cc:1196] T d45b67fb4faf4893acadd2a9cfd390f8 P b8a62c956c3e46baabad006ee1791b9d: Deleting WAL directory at /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-2/wal/wals/d45b67fb4faf4893acadd2a9cfd390f8
I20260923 23:17:28.959547 9388 ts_tablet_manager.cc:1939] T d45b67fb4faf4893acadd2a9cfd390f8 P b8a62c956c3e46baabad006ee1791b9d: Deleting consensus metadata
I20260923 23:17:28.960017 8888 catalog_manager.cc:4841] TS b8a62c956c3e46baabad006ee1791b9d (127.24.227.195:35361): tablet d45b67fb4faf4893acadd2a9cfd390f8 (table test-table [id=0ee292dbb36049759cb3bc7c27163a5a]) successfully deleted
I20260923 23:17:28.960711 9366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:17:28.960576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.193:46547 (local address 127.0.0.1:47458)
0923 23:17:28.960634 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0923 23:17:28.960674 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.193:46547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260923 23:17:28.960871 8886 catalog_manager.cc:4584] TS 9a094e6b7a64477fb4cda66abefcbdfd (127.24.227.193:46547): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d45b67fb4faf4893acadd2a9cfd390f8: Network error: Client connection negotiation failed: client connection to 127.24.227.193:46547: connect: Connection refused (error 111)
I20260923 23:17:28.962713 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 9231
I20260923 23:17:28.967125 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 8869
I20260923 23:17:48.972291 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.254:40121
--webserver_interface=127.24.227.254
--webserver_port=41419
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab
--principal=kudu/127.24.227.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:33557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.227.254:40121
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:17:49.050315 9391 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:17:49.050549 9391 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:17:49.050587 9391 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:17:49.051693 9391 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260923 23:17:49.051754 9391 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260923 23:17:49.051774 9391 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260923 23:17:49.051795 9391 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 23:17:49.051826 9391 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:17:49.051852 9391 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:17:49.051889 9391 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 23:17:49.051916 9391 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 23:17:49.053328 9391 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:33557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.24.227.254:40121
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab
--principal=kudu/127.24.227.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.254:40121
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.227.254
--webserver_port=41419
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9391
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:17:49.053593 9391 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:17:49.053817 9391 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:17:49.058420 9397 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:17:49.058401 9399 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:17:49.058442 9391 server_base.cc:1034] running on GCE node
W20260923 23:17:49.058418 9396 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:17:49.058917 9391 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:17:49.059134 9391 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:17:49.060261 9391 hybrid_clock.cc:648] HybridClock initialized: now 1790205469060249 us; error 28 us; skew 500 ppm
Sep 23 23:17:49 dist-test-slave-754n krb5kdc[8324](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205469, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:17:49.062618 9391 init.cc:377] Logged in from keytab as kudu/127.24.227.254@KRBTEST.COM (short username kudu)
I20260923 23:17:49.063023 9391 webserver.cc:460] Webserver started at http://127.24.227.254:41419/ using document root <none> and password file <none>
I20260923 23:17:49.063190 9391 fs_manager.cc:359] Metadata directory not provided
I20260923 23:17:49.063228 9391 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:17:49.064441 9391 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 23:17:49.064994 9406 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:17:49.065169 9391 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 23:17:49.065253 9391 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/master-0/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/master-0/wal
uuid: "70625401bbe84b8faa08091c510ffafe"
format_stamp: "Formatted at 2026-09-23 23:17:07 on dist-test-slave-754n"
server_key: "0315412c130e13af12293caf8c67a1a2"
server_key_iv: "61d15ddcff79eeabd9a47b62fdc13afd"
server_key_version: "encryptionkey@0"
I20260923 23:17:49.065569 9391 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:17:49.074589 9391 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:17:49.076359 9391 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:17:49.076507 9391 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:17:49.080175 9391 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.254:40121
I20260923 23:17:49.080205 9458 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.254:40121 every 8 connection(s)
I20260923 23:17:49.080552 9391 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
I20260923 23:17:49.081848 9459 sys_catalog.cc:263] Verifying existing consensus state
I20260923 23:17:49.082465 9459 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70625401bbe84b8faa08091c510ffafe: Bootstrap starting.
I20260923 23:17:49.086781 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 9391
I20260923 23:17:49.087050 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.227.193:46547
--local_ip_for_outbound_sockets=127.24.227.193
--tserver_master_addrs=127.24.227.254:40121
--webserver_port=37869
--webserver_interface=127.24.227.193
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.193.keytab
--principal=kudu/127.24.227.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:33557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 23:17:49.090987 9459 log.cc:824] T 00000000000000000000000000000000 P 70625401bbe84b8faa08091c510ffafe: Log is configured to *not* fsync() on all Append() calls
I20260923 23:17:49.093070 9459 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70625401bbe84b8faa08091c510ffafe: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 23:17:49.093467 9459 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70625401bbe84b8faa08091c510ffafe: Bootstrap complete.
I20260923 23:17:49.095577 9459 raft_consensus.cc:348] T 00000000000000000000000000000000 P 70625401bbe84b8faa08091c510ffafe [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70625401bbe84b8faa08091c510ffafe" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 40121 } }
I20260923 23:17:49.095814 9459 raft_consensus.cc:729] T 00000000000000000000000000000000 P 70625401bbe84b8faa08091c510ffafe [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70625401bbe84b8faa08091c510ffafe, State: Initialized, Role: FOLLOWER
I20260923 23:17:49.096002 9459 consensus_queue.cc:260] T 00000000000000000000000000000000 P 70625401bbe84b8faa08091c510ffafe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70625401bbe84b8faa08091c510ffafe" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 40121 } }
I20260923 23:17:49.096082 9459 raft_consensus.cc:388] T 00000000000000000000000000000000 P 70625401bbe84b8faa08091c510ffafe [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 23:17:49.096133 9459 raft_consensus.cc:482] T 00000000000000000000000000000000 P 70625401bbe84b8faa08091c510ffafe [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 23:17:49.096181 9459 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 70625401bbe84b8faa08091c510ffafe [term 2 FOLLOWER]: Advancing to term 3
I20260923 23:17:49.097069 9459 raft_consensus.cc:504] T 00000000000000000000000000000000 P 70625401bbe84b8faa08091c510ffafe [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70625401bbe84b8faa08091c510ffafe" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 40121 } }
I20260923 23:17:49.097178 9459 leader_election.cc:302] T 00000000000000000000000000000000 P 70625401bbe84b8faa08091c510ffafe [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 70625401bbe84b8faa08091c510ffafe; no voters:
I20260923 23:17:49.097364 9459 leader_election.cc:288] T 00000000000000000000000000000000 P 70625401bbe84b8faa08091c510ffafe [CANDIDATE]: Term 3 election: Requested vote from peers
I20260923 23:17:49.097394 9463 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 70625401bbe84b8faa08091c510ffafe [term 3 FOLLOWER]: Leader election won for term 3
I20260923 23:17:49.097499 9463 raft_consensus.cc:686] T 00000000000000000000000000000000 P 70625401bbe84b8faa08091c510ffafe [term 3 LEADER]: Becoming Leader. State: Replica: 70625401bbe84b8faa08091c510ffafe, State: Running, Role: LEADER
I20260923 23:17:49.097579 9463 consensus_queue.cc:237] T 00000000000000000000000000000000 P 70625401bbe84b8faa08091c510ffafe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70625401bbe84b8faa08091c510ffafe" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 40121 } }
I20260923 23:17:49.097612 9459 sys_catalog.cc:564] T 00000000000000000000000000000000 P 70625401bbe84b8faa08091c510ffafe [sys.catalog]: configured and running, proceeding with master startup.
I20260923 23:17:49.097807 9463 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70625401bbe84b8faa08091c510ffafe [sys.catalog]: SysCatalogTable state changed. Reason: New leader 70625401bbe84b8faa08091c510ffafe. Latest consensus state: current_term: 3 leader_uuid: "70625401bbe84b8faa08091c510ffafe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70625401bbe84b8faa08091c510ffafe" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 40121 } } }
I20260923 23:17:49.097918 9463 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70625401bbe84b8faa08091c510ffafe [sys.catalog]: This master's current role is: LEADER
I20260923 23:17:49.097810 9464 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70625401bbe84b8faa08091c510ffafe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "70625401bbe84b8faa08091c510ffafe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70625401bbe84b8faa08091c510ffafe" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 40121 } } }
I20260923 23:17:49.098075 9464 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70625401bbe84b8faa08091c510ffafe [sys.catalog]: This master's current role is: LEADER
I20260923 23:17:49.098168 9468 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 23:17:49.098961 9468 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 23:17:49.099067 9468 catalog_manager.cc:1259] Loaded cluster ID: f235dee3b5b04dd48ee9f905690e8ef1
I20260923 23:17:49.099112 9468 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 23:17:49.100255 9468 catalog_manager.cc:1504] Loading token signing keys...
I20260923 23:17:49.100419 9468 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 70625401bbe84b8faa08091c510ffafe: Loaded TSK: 0
I20260923 23:17:49.100466 9468 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 70625401bbe84b8faa08091c510ffafe: Loaded TSK: 1
F20260923 23:17:49.100667 9468 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fe00ebaa9ad google::LogMessage::Fail() at ??:0
@ 0x7fe00ebad88a google::LogMessage::SendToLog() at ??:0
@ 0x7fe00ebaa4de google::LogMessage::Flush() at ??:0
@ 0x7fe00ebaba19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fe011317104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fe011314d41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7fe016b041cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7fe016b0448e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7fe016b04594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7fe016acac48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7fe016ad4ad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fe010a780d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fe010a6d954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fe0118d16db start_thread at ??:0
@ 0x7fe00d6a861f clone at ??:0
W20260923 23:17:49.191618 9461 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:17:49.191895 9461 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:17:49.191932 9461 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:17:49.193476 9461 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:17:49.193550 9461 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:17:49.193585 9461 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.227.193
I20260923 23:17:49.195420 9461 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:33557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.193.keytab
--principal=kudu/127.24.227.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.227.193:46547
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.227.193
--webserver_port=37869
--tserver_master_addrs=127.24.227.254:40121
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9461
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.227.193
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:17:49.195631 9461 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:17:49.195875 9461 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:17:49.200492 9503 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:17:49.200556 9500 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:17:49.200739 9501 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:17:49.200984 9461 server_base.cc:1034] running on GCE node
I20260923 23:17:49.201171 9461 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:17:49.201407 9461 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:17:49.202551 9461 hybrid_clock.cc:648] HybridClock initialized: now 1790205469202536 us; error 31 us; skew 500 ppm
Sep 23 23:17:49 dist-test-slave-754n krb5kdc[8324](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205469, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:17:49.204891 9461 init.cc:377] Logged in from keytab as kudu/127.24.227.193@KRBTEST.COM (short username kudu)
I20260923 23:17:49.205287 9461 webserver.cc:460] Webserver started at http://127.24.227.193:37869/ using document root <none> and password file <none>
I20260923 23:17:49.205475 9461 fs_manager.cc:359] Metadata directory not provided
I20260923 23:17:49.205533 9461 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:17:49.206732 9461 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 23:17:49.207492 9510 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:17:49.207690 9461 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 23:17:49.207782 9461 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-0/wal
uuid: "9a094e6b7a64477fb4cda66abefcbdfd"
format_stamp: "Formatted at 2026-09-23 23:17:07 on dist-test-slave-754n"
server_key: "5fb09fd0f1d54d0105773590ccf76e2f"
server_key_iv: "f3b3e0fe422ee48e48d46a1ec2389715"
server_key_version: "encryptionkey@0"
I20260923 23:17:49.208132 9461 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:17:49.223003 9461 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:17:49.225121 9461 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:17:49.225261 9461 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:17:49.225461 9461 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 23:17:49.225847 9517 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260923 23:17:49.227634 9461 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260923 23:17:49.227712 9461 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260923 23:17:49.227751 9461 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260923 23:17:49.228477 9461 ts_tablet_manager.cc:610] Registered 1 tablets
I20260923 23:17:49.228524 9461 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260923 23:17:49.228606 9517 tablet_bootstrap.cc:492] T d45b67fb4faf4893acadd2a9cfd390f8 P 9a094e6b7a64477fb4cda66abefcbdfd: Bootstrap starting.
I20260923 23:17:49.230345 9517 log.cc:824] T d45b67fb4faf4893acadd2a9cfd390f8 P 9a094e6b7a64477fb4cda66abefcbdfd: Log is configured to *not* fsync() on all Append() calls
I20260923 23:17:49.233124 9517 tablet_bootstrap.cc:492] T d45b67fb4faf4893acadd2a9cfd390f8 P 9a094e6b7a64477fb4cda66abefcbdfd: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 23:17:49.233424 9517 tablet_bootstrap.cc:492] T d45b67fb4faf4893acadd2a9cfd390f8 P 9a094e6b7a64477fb4cda66abefcbdfd: Bootstrap complete.
I20260923 23:17:49.233572 9517 ts_tablet_manager.cc:1397] T d45b67fb4faf4893acadd2a9cfd390f8 P 9a094e6b7a64477fb4cda66abefcbdfd: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260923 23:17:49.235380 9461 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.193:46547
I20260923 23:17:49.235464 9625 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.193:46547 every 8 connection(s)
I20260923 23:17:49.235296 9517 raft_consensus.cc:348] T d45b67fb4faf4893acadd2a9cfd390f8 P 9a094e6b7a64477fb4cda66abefcbdfd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ac6b50428b346a1b5604b42baf02cee" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 36765 } } peers { permanent_uuid: "b8a62c956c3e46baabad006ee1791b9d" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 35361 } } peers { permanent_uuid: "9a094e6b7a64477fb4cda66abefcbdfd" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 46547 } }
I20260923 23:17:49.235605 9517 raft_consensus.cc:729] T d45b67fb4faf4893acadd2a9cfd390f8 P 9a094e6b7a64477fb4cda66abefcbdfd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a094e6b7a64477fb4cda66abefcbdfd, State: Initialized, Role: FOLLOWER
I20260923 23:17:49.235731 9461 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-0/data/info.pb
I20260923 23:17:49.235757 9517 consensus_queue.cc:260] T d45b67fb4faf4893acadd2a9cfd390f8 P 9a094e6b7a64477fb4cda66abefcbdfd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ac6b50428b346a1b5604b42baf02cee" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 36765 } } peers { permanent_uuid: "b8a62c956c3e46baabad006ee1791b9d" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 35361 } } peers { permanent_uuid: "9a094e6b7a64477fb4cda66abefcbdfd" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 46547 } }
I20260923 23:17:49.236014 9517 ts_tablet_manager.cc:1428] T d45b67fb4faf4893acadd2a9cfd390f8 P 9a094e6b7a64477fb4cda66abefcbdfd: Time spent starting tablet: real 0.002s user 0.000s sys 0.004s
I20260923 23:17:49.242136 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 9461
I20260923 23:17:49.242384 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.227.194:36765
--local_ip_for_outbound_sockets=127.24.227.194
--tserver_master_addrs=127.24.227.254:40121
--webserver_port=34375
--webserver_interface=127.24.227.194
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.194.keytab
--principal=kudu/127.24.227.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:33557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:17:49.319476 9629 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 23:17:49.236455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:40121 (local address 127.24.227.193:51501)
0923 23:17:49.236636 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0923 23:17:49.236672 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0923 23:17:49.236813 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:17:49.319261 (+ 82448us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:40121: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":90,"threads_started":1}
W20260923 23:17:49.319749 9626 heartbeater.cc:644] Failed to heartbeat to 127.24.227.254:40121 (0 consecutive failures): Network error: Failed to ping master at 127.24.227.254:40121: Client connection negotiation failed: client connection to 127.24.227.254:40121: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260923 23:17:49.320097 9629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:17:49.319967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:40121 (local address 127.24.227.193:37875)
0923 23:17:49.320009 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0923 23:17:49.320065 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:40121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260923 23:17:49.320458 9629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:17:49.320366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:40121 (local address 127.24.227.193:34605)
0923 23:17:49.320411 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0923 23:17:49.320437 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:40121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260923 23:17:49.320575 9626 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260923 23:17:49.341109 9630 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:17:49.341272 9630 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:17:49.341295 9630 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:17:49.342289 9630 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:17:49.342324 9630 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:17:49.342347 9630 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.227.194
I20260923 23:17:49.343622 9630 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:33557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.194.keytab
--principal=kudu/127.24.227.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.227.194:36765
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.227.194
--webserver_port=34375
--tserver_master_addrs=127.24.227.254:40121
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9630
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.227.194
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:17:49.343778 9630 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:17:49.343990 9630 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:17:49.348613 9635 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:17:49.348650 9636 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:17:49.348668 9630 server_base.cc:1034] running on GCE node
W20260923 23:17:49.348635 9638 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:17:49.349004 9630 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:17:49.349242 9630 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:17:49.350378 9630 hybrid_clock.cc:648] HybridClock initialized: now 1790205469350368 us; error 28 us; skew 500 ppm
Sep 23 23:17:49 dist-test-slave-754n krb5kdc[8324](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205469, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:17:49.352852 9630 init.cc:377] Logged in from keytab as kudu/127.24.227.194@KRBTEST.COM (short username kudu)
I20260923 23:17:49.353229 9630 webserver.cc:460] Webserver started at http://127.24.227.194:34375/ using document root <none> and password file <none>
I20260923 23:17:49.353443 9630 fs_manager.cc:359] Metadata directory not provided
I20260923 23:17:49.353507 9630 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:17:49.354748 9630 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 23:17:49.355253 9645 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:17:49.355403 9630 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 23:17:49.355494 9630 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-1/wal
uuid: "5ac6b50428b346a1b5604b42baf02cee"
format_stamp: "Formatted at 2026-09-23 23:17:08 on dist-test-slave-754n"
server_key: "2cc6b3f7ca7a86612a83fcbd7ba6a005"
server_key_iv: "463e05b2ac27eb65958963ee9210a15c"
server_key_version: "encryptionkey@0"
I20260923 23:17:49.355799 9630 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:17:49.393280 9630 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:17:49.395462 9630 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:17:49.395630 9630 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:17:49.395838 9630 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 23:17:49.396139 9630 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 23:17:49.396190 9630 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 23:17:49.396229 9630 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 23:17:49.396255 9630 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 23:17:49.402053 9630 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.194:36765
I20260923 23:17:49.402130 9758 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.194:36765 every 8 connection(s)
I20260923 23:17:49.402444 9630 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-1/data/info.pb
I20260923 23:17:49.403414 9761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:17:49.403085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:40121 (local address 127.24.227.194:40251)
0923 23:17:49.403228 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0923 23:17:49.403288 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:40121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":69,"threads_started":1}
W20260923 23:17:49.403817 9759 heartbeater.cc:644] Failed to heartbeat to 127.24.227.254:40121 (0 consecutive failures): Network error: Failed to ping master at 127.24.227.254:40121: Client connection negotiation failed: client connection to 127.24.227.254:40121: connect: Connection refused (error 111)
I20260923 23:17:49.404103 9761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:17:49.404013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:40121 (local address 127.24.227.194:54461)
0923 23:17:49.404055 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0923 23:17:49.404084 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:40121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260923 23:17:49.404399 9761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:17:49.404327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:40121 (local address 127.24.227.194:42373)
0923 23:17:49.404363 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0923 23:17:49.404385 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:40121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260923 23:17:49.404524 9759 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260923 23:17:49.408061 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 9630
I20260923 23:17:49.408313 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.227.195:35361
--local_ip_for_outbound_sockets=127.24.227.195
--tserver_master_addrs=127.24.227.254:40121
--webserver_port=38179
--webserver_interface=127.24.227.195
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.195.keytab
--principal=kudu/127.24.227.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:33557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:17:49.489719 9762 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:17:49.489893 9762 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:17:49.489916 9762 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:17:49.491050 9762 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:17:49.491092 9762 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:17:49.491146 9762 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.227.195
I20260923 23:17:49.492604 9762 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:33557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.195.keytab
--principal=kudu/127.24.227.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.227.195:35361
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.227.195
--webserver_port=38179
--tserver_master_addrs=127.24.227.254:40121
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9762
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.227.195
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:17:49.492820 9762 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:17:49.493026 9762 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:17:49.497782 9770 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:17:49.497818 9767 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:17:49.497953 9762 server_base.cc:1034] running on GCE node
W20260923 23:17:49.497794 9768 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:17:49.498235 9762 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:17:49.498469 9762 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:17:49.499622 9762 hybrid_clock.cc:648] HybridClock initialized: now 1790205469499613 us; error 28 us; skew 500 ppm
Sep 23 23:17:49 dist-test-slave-754n krb5kdc[8324](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205469, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:17:49.501935 9762 init.cc:377] Logged in from keytab as kudu/127.24.227.195@KRBTEST.COM (short username kudu)
I20260923 23:17:49.502381 9762 webserver.cc:460] Webserver started at http://127.24.227.195:38179/ using document root <none> and password file <none>
I20260923 23:17:49.502638 9762 fs_manager.cc:359] Metadata directory not provided
I20260923 23:17:49.502707 9762 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:17:49.503974 9762 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 23:17:49.504540 9777 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:17:49.504709 9762 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 23:17:49.504797 9762 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-2/wal
uuid: "b8a62c956c3e46baabad006ee1791b9d"
format_stamp: "Formatted at 2026-09-23 23:17:08 on dist-test-slave-754n"
server_key: "63c6ad468db4723a9a3d64de89590129"
server_key_iv: "4728729873838112e0e61078612ec59c"
server_key_version: "encryptionkey@0"
I20260923 23:17:49.505098 9762 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:17:49.518754 9762 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:17:49.520632 9762 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:17:49.520792 9762 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:17:49.521023 9762 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 23:17:49.521320 9762 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 23:17:49.521374 9762 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 23:17:49.521415 9762 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 23:17:49.521441 9762 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 23:17:49.527177 9762 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.195:35361
I20260923 23:17:49.527221 9890 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.195:35361 every 8 connection(s)
I20260923 23:17:49.527566 9762 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790205259943518-25487-0/minicluster-data/ts-2/data/info.pb
I20260923 23:17:49.528658 9893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:17:49.528278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:40121 (local address 127.24.227.195:57359)
0923 23:17:49.528428 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0923 23:17:49.528497 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:40121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":72,"threads_started":1}
W20260923 23:17:49.529038 9891 heartbeater.cc:644] Failed to heartbeat to 127.24.227.254:40121 (0 consecutive failures): Network error: Failed to ping master at 127.24.227.254:40121: Client connection negotiation failed: client connection to 127.24.227.254:40121: connect: Connection refused (error 111)
I20260923 23:17:49.529301 9893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:17:49.529216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:40121 (local address 127.24.227.195:43709)
0923 23:17:49.529260 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0923 23:17:49.529285 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:40121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260923 23:17:49.529536 9893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:17:49.529486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:40121 (local address 127.24.227.195:52489)
0923 23:17:49.529507 (+ 21us) negotiation.cc:107] Waiting for socket to connect
0923 23:17:49.529524 (+ 17us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:40121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12}
W20260923 23:17:49.529680 9891 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260923 23:17:49.532737 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 9762
I20260923 23:17:50.321092 9895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:17:50.320869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:40121 (local address 127.24.227.193:46515)
0923 23:17:50.320980 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0923 23:17:50.321014 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:40121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":56,"threads_started":1}
I20260923 23:17:50.404985 9896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:17:50.404802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:40121 (local address 127.24.227.194:40543)
0923 23:17:50.404907 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0923 23:17:50.404938 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:40121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":54,"threads_started":1}
I20260923 23:17:50.477793 9897 raft_consensus.cc:482] T d45b67fb4faf4893acadd2a9cfd390f8 P 9a094e6b7a64477fb4cda66abefcbdfd [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 23:17:50.477927 9897 raft_consensus.cc:504] T d45b67fb4faf4893acadd2a9cfd390f8 P 9a094e6b7a64477fb4cda66abefcbdfd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ac6b50428b346a1b5604b42baf02cee" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 36765 } } peers { permanent_uuid: "b8a62c956c3e46baabad006ee1791b9d" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 35361 } } peers { permanent_uuid: "9a094e6b7a64477fb4cda66abefcbdfd" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 46547 } }
I20260923 23:17:50.478224 9897 leader_election.cc:288] T d45b67fb4faf4893acadd2a9cfd390f8 P 9a094e6b7a64477fb4cda66abefcbdfd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5ac6b50428b346a1b5604b42baf02cee (127.24.227.194:36765), b8a62c956c3e46baabad006ee1791b9d (127.24.227.195:35361)
Sep 23 23:17:50 dist-test-slave-754n krb5kdc[8324](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205469, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.193@KRBTEST.COM for kudu/127.24.227.195@KRBTEST.COM
Sep 23 23:17:50 dist-test-slave-754n krb5kdc[8324](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205469, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.193@KRBTEST.COM for kudu/127.24.227.194@KRBTEST.COM
I20260923 23:17:50.488946 9900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:17:50.478524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.193:57651 (local address 127.24.227.195:35361)
0923 23:17:50.478686 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0923 23:17:50.478693 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 23:17:50.478712 (+ 19us) server_negotiation.cc:407] Connection header received
0923 23:17:50.479261 (+ 549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:17:50.479265 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:17:50.479329 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:17:50.479483 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:17:50.480602 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:17:50.481442 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:17:50.482443 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:17:50.482838 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:17:50.482970 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:17:50.485358 (+ 2388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:17:50.485361 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:17:50.485362 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:17:50.485371 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:17:50.487293 (+ 1922us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:17:50.487714 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:17:50.487716 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:17:50.487717 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:17:50.487808 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:17:50.488098 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:17:50.488099 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:17:50.488099 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:17:50.488332 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:17:50.488422 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0923 23:17:50.488713 (+ 291us) server_negotiation.cc:299] Negotiation successful
0923 23:17:50.488837 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":52,"threads_started":1}
I20260923 23:17:50.488961 9899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:17:50.478499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.193:32875 (local address 127.24.227.194:36765)
0923 23:17:50.478633 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0923 23:17:50.478648 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0923 23:17:50.478712 (+ 64us) server_negotiation.cc:407] Connection header received
0923 23:17:50.479264 (+ 552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:17:50.479267 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:17:50.479325 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:17:50.479577 (+ 252us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:17:50.480401 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:17:50.481417 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:17:50.482428 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:17:50.482840 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:17:50.482959 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:17:50.485859 (+ 2900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:17:50.485880 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:17:50.485881 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:17:50.485889 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:17:50.487517 (+ 1628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:17:50.487887 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:17:50.487890 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:17:50.487890 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:17:50.487982 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:17:50.488200 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:17:50.488201 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:17:50.488201 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:17:50.488412 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:17:50.488497 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0923 23:17:50.488705 (+ 208us) server_negotiation.cc:299] Negotiation successful
0923 23:17:50.488842 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97,"thread_start_us":72,"threads_started":1}
I20260923 23:17:50.488963 9895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:17:50.478347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.195:35361 (local address 127.24.227.193:57651)
0923 23:17:50.478469 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0923 23:17:50.478480 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 23:17:50.478539 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:17:50.479567 (+ 1028us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:17:50.479567 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:17:50.480028 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:17:50.480480 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:17:50.480483 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:17:50.481570 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:17:50.481571 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:17:50.482332 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:17:50.482334 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:17:50.482937 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:17:50.482938 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:17:50.483061 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:17:50.483689 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:17:50.483693 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:17:50.485235 (+ 1542us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:17:50.487390 (+ 2155us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:17:50.487397 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:17:50.487399 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:17:50.487619 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:17:50.487897 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:17:50.487898 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:17:50.487898 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:17:50.488009 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:17:50.488415 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:17:50.488421 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:17:50.488548 (+ 127us) client_negotiation.cc:769] Sending connection context
0923 23:17:50.488645 (+ 97us) client_negotiation.cc:240] Negotiation successful
0923 23:17:50.488827 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69,"mutex_wait_us":58}
I20260923 23:17:50.489377 9898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:17:50.478433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.194:36765 (local address 127.24.227.193:32875)
0923 23:17:50.478666 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0923 23:17:50.478673 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0923 23:17:50.478710 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:17:50.479550 (+ 840us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:17:50.479553 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:17:50.480037 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:17:50.480260 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:17:50.480267 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:17:50.481519 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:17:50.481521 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:17:50.482334 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:17:50.482336 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:17:50.482935 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:17:50.482936 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:17:50.483061 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:17:50.483627 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:17:50.483635 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:17:50.485751 (+ 2116us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:17:50.487609 (+ 1858us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:17:50.487610 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:17:50.487611 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:17:50.487786 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:17:50.488070 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:17:50.488071 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:17:50.488071 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:17:50.488135 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:17:50.488492 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:17:50.488493 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:17:50.488559 (+ 66us) client_negotiation.cc:769] Sending connection context
0923 23:17:50.488648 (+ 89us) client_negotiation.cc:240] Negotiation successful
0923 23:17:50.488904 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218,"mutex_wait_us":134,"thread_start_us":66,"threads_started":1}
W20260923 23:17:50.489882 9513 leader_election.cc:341] T d45b67fb4faf4893acadd2a9cfd390f8 P 9a094e6b7a64477fb4cda66abefcbdfd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b8a62c956c3e46baabad006ee1791b9d (127.24.227.195:35361): Not found: Tablet not found: d45b67fb4faf4893acadd2a9cfd390f8
W20260923 23:17:50.490063 9514 leader_election.cc:341] T d45b67fb4faf4893acadd2a9cfd390f8 P 9a094e6b7a64477fb4cda66abefcbdfd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5ac6b50428b346a1b5604b42baf02cee (127.24.227.194:36765): Not found: Tablet not found: d45b67fb4faf4893acadd2a9cfd390f8
I20260923 23:17:50.490114 9514 leader_election.cc:302] T d45b67fb4faf4893acadd2a9cfd390f8 P 9a094e6b7a64477fb4cda66abefcbdfd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9a094e6b7a64477fb4cda66abefcbdfd; no voters: 5ac6b50428b346a1b5604b42baf02cee, b8a62c956c3e46baabad006ee1791b9d
I20260923 23:17:50.490243 9897 raft_consensus.cc:2726] T d45b67fb4faf4893acadd2a9cfd390f8 P 9a094e6b7a64477fb4cda66abefcbdfd [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260923 23:17:50.530153 9901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:17:50.529965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:40121 (local address 127.24.227.195:53649)
0923 23:17:50.530085 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0923 23:17:50.530122 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:40121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":62,"threads_started":1}
I20260923 23:17:51.321728 9902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:17:51.321533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:40121 (local address 127.24.227.193:35937)
0923 23:17:51.321651 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0923 23:17:51.321683 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:40121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":60,"threads_started":1}
I20260923 23:17:51.405556 9903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:17:51.405367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:40121 (local address 127.24.227.194:46397)
0923 23:17:51.405488 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0923 23:17:51.405520 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:40121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":71,"threads_started":1}
I20260923 23:17:51.530823 9904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:17:51.530591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:40121 (local address 127.24.227.195:39285)
0923 23:17:51.530727 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0923 23:17:51.530757 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:40121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":83,"threads_started":1}
I20260923 23:17:52.322387 9905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:17:52.322172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:40121 (local address 127.24.227.193:40763)
0923 23:17:52.322315 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0923 23:17:52.322344 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:40121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":69,"threads_started":1}
I20260923 23:17:52.380901 9906 raft_consensus.cc:482] T d45b67fb4faf4893acadd2a9cfd390f8 P 9a094e6b7a64477fb4cda66abefcbdfd [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 23:17:52.380988 9906 raft_consensus.cc:504] T d45b67fb4faf4893acadd2a9cfd390f8 P 9a094e6b7a64477fb4cda66abefcbdfd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ac6b50428b346a1b5604b42baf02cee" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 36765 } } peers { permanent_uuid: "b8a62c956c3e46baabad006ee1791b9d" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 35361 } } peers { permanent_uuid: "9a094e6b7a64477fb4cda66abefcbdfd" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 46547 } }
I20260923 23:17:52.381193 9906 leader_election.cc:288] T d45b67fb4faf4893acadd2a9cfd390f8 P 9a094e6b7a64477fb4cda66abefcbdfd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5ac6b50428b346a1b5604b42baf02cee (127.24.227.194:36765), b8a62c956c3e46baabad006ee1791b9d (127.24.227.195:35361)
W20260923 23:17:52.381594 9513 leader_election.cc:341] T d45b67fb4faf4893acadd2a9cfd390f8 P 9a094e6b7a64477fb4cda66abefcbdfd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b8a62c956c3e46baabad006ee1791b9d (127.24.227.195:35361): Not found: Tablet not found: d45b67fb4faf4893acadd2a9cfd390f8
W20260923 23:17:52.381690 9514 leader_election.cc:341] T d45b67fb4faf4893acadd2a9cfd390f8 P 9a094e6b7a64477fb4cda66abefcbdfd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5ac6b50428b346a1b5604b42baf02cee (127.24.227.194:36765): Not found: Tablet not found: d45b67fb4faf4893acadd2a9cfd390f8
I20260923 23:17:52.381779 9514 leader_election.cc:302] T d45b67fb4faf4893acadd2a9cfd390f8 P 9a094e6b7a64477fb4cda66abefcbdfd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9a094e6b7a64477fb4cda66abefcbdfd; no voters: 5ac6b50428b346a1b5604b42baf02cee, b8a62c956c3e46baabad006ee1791b9d
I20260923 23:17:52.381850 9906 raft_consensus.cc:2726] T d45b67fb4faf4893acadd2a9cfd390f8 P 9a094e6b7a64477fb4cda66abefcbdfd [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260923 23:17:52.406095 9907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:17:52.405904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:40121 (local address 127.24.227.194:43125)
0923 23:17:52.406034 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0923 23:17:52.406062 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:40121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":65,"threads_started":1}
I20260923 23:17:52.531410 9908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:17:52.531211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:40121 (local address 127.24.227.195:52095)
0923 23:17:52.531339 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0923 23:17:52.531375 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:40121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":57,"threads_started":1}
I20260923 23:17:53.323060 9909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:17:53.322814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:40121 (local address 127.24.227.193:39851)
0923 23:17:53.322961 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0923 23:17:53.323006 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:40121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":69,"threads_started":1}
I20260923 23:17:53.406679 9910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:17:53.406484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:40121 (local address 127.24.227.194:40245)
0923 23:17:53.406615 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0923 23:17:53.406645 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:40121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":70,"threads_started":1}
I20260923 23:17:53.532039 9911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:17:53.531830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:40121 (local address 127.24.227.195:45117)
0923 23:17:53.531974 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0923 23:17:53.532004 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:40121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":97,"threads_started":1}
I20260923 23:17:54.206413 9912 raft_consensus.cc:482] T d45b67fb4faf4893acadd2a9cfd390f8 P 9a094e6b7a64477fb4cda66abefcbdfd [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 23:17:54.206521 9912 raft_consensus.cc:504] T d45b67fb4faf4893acadd2a9cfd390f8 P 9a094e6b7a64477fb4cda66abefcbdfd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ac6b50428b346a1b5604b42baf02cee" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 36765 } } peers { permanent_uuid: "b8a62c956c3e46baabad006ee1791b9d" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 35361 } } peers { permanent_uuid: "9a094e6b7a64477fb4cda66abefcbdfd" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 46547 } }
I20260923 23:17:54.206705 9912 leader_election.cc:288] T d45b67fb4faf4893acadd2a9cfd390f8 P 9a094e6b7a64477fb4cda66abefcbdfd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5ac6b50428b346a1b5604b42baf02cee (127.24.227.194:36765), b8a62c956c3e46baabad006ee1791b9d (127.24.227.195:35361)
W20260923 23:17:54.207073 9513 leader_election.cc:341] T d45b67fb4faf4893acadd2a9cfd390f8 P 9a094e6b7a64477fb4cda66abefcbdfd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b8a62c956c3e46baabad006ee1791b9d (127.24.227.195:35361): Not found: Tablet not found: d45b67fb4faf4893acadd2a9cfd390f8
W20260923 23:17:54.207146 9514 leader_election.cc:341] T d45b67fb4faf4893acadd2a9cfd390f8 P 9a094e6b7a64477fb4cda66abefcbdfd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5ac6b50428b346a1b5604b42baf02cee (127.24.227.194:36765): Not found: Tablet not found: d45b67fb4faf4893acadd2a9cfd390f8
I20260923 23:17:54.207175 9514 leader_election.cc:302] T d45b67fb4faf4893acadd2a9cfd390f8 P 9a094e6b7a64477fb4cda66abefcbdfd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9a094e6b7a64477fb4cda66abefcbdfd; no voters: 5ac6b50428b346a1b5604b42baf02cee, b8a62c956c3e46baabad006ee1791b9d
I20260923 23:17:54.207243 9912 raft_consensus.cc:2726] T d45b67fb4faf4893acadd2a9cfd390f8 P 9a094e6b7a64477fb4cda66abefcbdfd [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260923 23:17:54.323693 9913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:17:54.323485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:40121 (local address 127.24.227.193:37207)
0923 23:17:54.323618 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0923 23:17:54.323650 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:40121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":72,"threads_started":1}
I20260923 23:17:54.407296 9914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:17:54.407108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:40121 (local address 127.24.227.194:34725)
0923 23:17:54.407226 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0923 23:17:54.407260 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:40121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":61,"threads_started":1}
I20260923 23:17:54.532620 9915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:17:54.532416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:40121 (local address 127.24.227.195:34173)
0923 23:17:54.532549 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0923 23:17:54.532583 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:40121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":63,"threads_started":1}
I20260923 23:17:55.324292 9916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:17:55.324096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:40121 (local address 127.24.227.193:53369)
0923 23:17:55.324217 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0923 23:17:55.324249 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:40121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":70,"threads_started":1}
I20260923 23:17:55.407946 9917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:17:55.407724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:40121 (local address 127.24.227.194:36615)
0923 23:17:55.407842 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0923 23:17:55.407906 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:40121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":68,"threads_started":1}
I20260923 23:17:55.533197 9918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:17:55.533003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:40121 (local address 127.24.227.195:45057)
0923 23:17:55.533126 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0923 23:17:55.533159 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:40121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":69,"threads_started":1}
I20260923 23:17:56.324930 9919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:17:56.324717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:40121 (local address 127.24.227.193:33607)
0923 23:17:56.324844 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0923 23:17:56.324882 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:40121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":72,"threads_started":1}
I20260923 23:17:56.408491 9920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:17:56.408294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:40121 (local address 127.24.227.194:46049)
0923 23:17:56.408416 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0923 23:17:56.408450 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:40121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":66,"threads_started":1}
I20260923 23:17:56.533892 9921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:17:56.533683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:40121 (local address 127.24.227.195:35715)
0923 23:17:56.533822 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0923 23:17:56.533855 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:40121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":66,"threads_started":1}
I20260923 23:17:57.325654 9922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:17:57.325455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:40121 (local address 127.24.227.193:51493)
0923 23:17:57.325579 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0923 23:17:57.325611 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:40121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":71,"threads_started":1}
I20260923 23:17:57.409133 9923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:17:57.408947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:40121 (local address 127.24.227.194:33665)
0923 23:17:57.409067 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0923 23:17:57.409097 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:40121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":67,"threads_started":1}
I20260923 23:17:57.534487 9924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:17:57.534287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:40121 (local address 127.24.227.195:35177)
0923 23:17:57.534408 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0923 23:17:57.534442 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:40121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":56,"threads_started":1}
I20260923 23:17:58.326282 9925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:17:58.326078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:40121 (local address 127.24.227.193:54109)
0923 23:17:58.326205 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0923 23:17:58.326238 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:40121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":68,"threads_started":1}
I20260923 23:17:58.409752 9926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:17:58.409517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:40121 (local address 127.24.227.194:44559)
0923 23:17:58.409647 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0923 23:17:58.409682 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:40121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":55,"threads_started":1}
I20260923 23:17:58.535092 9927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:17:58.534859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:40121 (local address 127.24.227.195:41515)
0923 23:17:58.535012 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0923 23:17:58.535045 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:40121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":68,"threads_started":1}
I20260923 23:17:58.852351 9928 raft_consensus.cc:482] T d45b67fb4faf4893acadd2a9cfd390f8 P 9a094e6b7a64477fb4cda66abefcbdfd [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 23:17:58.852458 9928 raft_consensus.cc:504] T d45b67fb4faf4893acadd2a9cfd390f8 P 9a094e6b7a64477fb4cda66abefcbdfd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ac6b50428b346a1b5604b42baf02cee" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 36765 } } peers { permanent_uuid: "b8a62c956c3e46baabad006ee1791b9d" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 35361 } } peers { permanent_uuid: "9a094e6b7a64477fb4cda66abefcbdfd" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 46547 } }
I20260923 23:17:58.852662 9928 leader_election.cc:288] T d45b67fb4faf4893acadd2a9cfd390f8 P 9a094e6b7a64477fb4cda66abefcbdfd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5ac6b50428b346a1b5604b42baf02cee (127.24.227.194:36765), b8a62c956c3e46baabad006ee1791b9d (127.24.227.195:35361)
W20260923 23:17:58.853008 9513 leader_election.cc:341] T d45b67fb4faf4893acadd2a9cfd390f8 P 9a094e6b7a64477fb4cda66abefcbdfd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b8a62c956c3e46baabad006ee1791b9d (127.24.227.195:35361): Not found: Tablet not found: d45b67fb4faf4893acadd2a9cfd390f8
W20260923 23:17:58.853094 9514 leader_election.cc:341] T d45b67fb4faf4893acadd2a9cfd390f8 P 9a094e6b7a64477fb4cda66abefcbdfd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5ac6b50428b346a1b5604b42baf02cee (127.24.227.194:36765): Not found: Tablet not found: d45b67fb4faf4893acadd2a9cfd390f8
I20260923 23:17:58.853186 9514 leader_election.cc:302] T d45b67fb4faf4893acadd2a9cfd390f8 P 9a094e6b7a64477fb4cda66abefcbdfd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9a094e6b7a64477fb4cda66abefcbdfd; no voters: 5ac6b50428b346a1b5604b42baf02cee, b8a62c956c3e46baabad006ee1791b9d
I20260923 23:17:58.853276 9928 raft_consensus.cc:2726] T d45b67fb4faf4893acadd2a9cfd390f8 P 9a094e6b7a64477fb4cda66abefcbdfd [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260923 23:17:59.326961 9929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:17:59.326697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:40121 (local address 127.24.227.193:47463)
0923 23:17:59.326845 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0923 23:17:59.326882 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:40121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":73,"threads_started":1}
I20260923 23:17:59.410344 9930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:17:59.410144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:40121 (local address 127.24.227.194:51805)
0923 23:17:59.410275 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0923 23:17:59.410305 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:40121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":68,"threads_started":1}
I20260923 23:17:59.535686 9931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:17:59.535489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:40121 (local address 127.24.227.195:45097)
0923 23:17:59.535621 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0923 23:17:59.535651 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:40121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":68,"threads_started":1}
I20260923 23:18:00.327607 9932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:00.327412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:40121 (local address 127.24.227.193:53525)
0923 23:18:00.327527 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:00.327560 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:40121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":61,"threads_started":1}
I20260923 23:18:00.411027 9933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:00.410732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:40121 (local address 127.24.227.194:35177)
0923 23:18:00.410939 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:00.410985 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:40121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":100,"threads_started":1}
I20260923 23:18:00.536275 9934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:00.536090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:40121 (local address 127.24.227.195:45631)
0923 23:18:00.536204 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:00.536237 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:40121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":55,"threads_started":1}
I20260923 23:18:01.328212 9935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:01.328026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:40121 (local address 127.24.227.193:52727)
0923 23:18:01.328138 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:01.328168 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:40121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":58,"threads_started":1}
I20260923 23:18:01.411645 9936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:01.411449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:40121 (local address 127.24.227.194:58905)
0923 23:18:01.411575 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:01.411608 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:40121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":55,"threads_started":1}
I20260923 23:18:01.536892 9937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:01.536705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:40121 (local address 127.24.227.195:53131)
0923 23:18:01.536816 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:01.536850 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:40121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":56,"threads_started":1}
I20260923 23:18:02.328840 9938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:02.328651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:40121 (local address 127.24.227.193:36667)
0923 23:18:02.328762 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:02.328795 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:40121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":58,"threads_started":1}
I20260923 23:18:02.412262 9939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:02.412061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:40121 (local address 127.24.227.194:42493)
0923 23:18:02.412178 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:02.412214 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:40121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":56,"threads_started":1}
I20260923 23:18:02.537540 9940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:02.537288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:40121 (local address 127.24.227.195:38071)
0923 23:18:02.537452 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:02.537488 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:40121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":70,"threads_started":1}
I20260923 23:18:03.329447 9941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:03.329246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:40121 (local address 127.24.227.193:43595)
0923 23:18:03.329368 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:03.329401 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:40121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":71,"threads_started":1}
I20260923 23:18:03.412811 9942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:03.412644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:40121 (local address 127.24.227.194:45169)
0923 23:18:03.412748 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:03.412778 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:40121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":55,"threads_started":1}
I20260923 23:18:03.538309 9943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:03.538113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:40121 (local address 127.24.227.195:57845)
0923 23:18:03.538237 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:03.538270 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:40121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":61,"threads_started":1}
I20260923 23:18:04.330200 9944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:04.329938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:40121 (local address 127.24.227.193:37531)
0923 23:18:04.330123 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:04.330157 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:40121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":55,"threads_started":1}
I20260923 23:18:04.413388 9945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:04.413216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:40121 (local address 127.24.227.194:33241)
0923 23:18:04.413330 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:04.413356 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:40121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":70,"threads_started":1}
I20260923 23:18:04.486685 9946 raft_consensus.cc:482] T d45b67fb4faf4893acadd2a9cfd390f8 P 9a094e6b7a64477fb4cda66abefcbdfd [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 23:18:04.486783 9946 raft_consensus.cc:504] T d45b67fb4faf4893acadd2a9cfd390f8 P 9a094e6b7a64477fb4cda66abefcbdfd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ac6b50428b346a1b5604b42baf02cee" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 36765 } } peers { permanent_uuid: "b8a62c956c3e46baabad006ee1791b9d" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 35361 } } peers { permanent_uuid: "9a094e6b7a64477fb4cda66abefcbdfd" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 46547 } }
I20260923 23:18:04.486928 9946 leader_election.cc:288] T d45b67fb4faf4893acadd2a9cfd390f8 P 9a094e6b7a64477fb4cda66abefcbdfd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5ac6b50428b346a1b5604b42baf02cee (127.24.227.194:36765), b8a62c956c3e46baabad006ee1791b9d (127.24.227.195:35361)
W20260923 23:18:04.487319 9513 leader_election.cc:341] T d45b67fb4faf4893acadd2a9cfd390f8 P 9a094e6b7a64477fb4cda66abefcbdfd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b8a62c956c3e46baabad006ee1791b9d (127.24.227.195:35361): Not found: Tablet not found: d45b67fb4faf4893acadd2a9cfd390f8
W20260923 23:18:04.487443 9514 leader_election.cc:341] T d45b67fb4faf4893acadd2a9cfd390f8 P 9a094e6b7a64477fb4cda66abefcbdfd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5ac6b50428b346a1b5604b42baf02cee (127.24.227.194:36765): Not found: Tablet not found: d45b67fb4faf4893acadd2a9cfd390f8
I20260923 23:18:04.487483 9514 leader_election.cc:302] T d45b67fb4faf4893acadd2a9cfd390f8 P 9a094e6b7a64477fb4cda66abefcbdfd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9a094e6b7a64477fb4cda66abefcbdfd; no voters: 5ac6b50428b346a1b5604b42baf02cee, b8a62c956c3e46baabad006ee1791b9d
I20260923 23:18:04.487562 9946 raft_consensus.cc:2726] T d45b67fb4faf4893acadd2a9cfd390f8 P 9a094e6b7a64477fb4cda66abefcbdfd [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260923 23:18:04.538981 9947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:04.538787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:40121 (local address 127.24.227.195:37279)
0923 23:18:04.538903 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:04.538933 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.254:40121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":57,"threads_started":1}
I20260923 23:18:04.542456 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 9461
I20260923 23:18:04.547169 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 9630
I20260923 23:18:04.551271 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 9762
2026-09-23T23:18:04Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58089 ms)
[----------] 30 tests from SecurityITest (224615 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790205259943518-25487-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 23:18:04 dist-test-slave-754n krb5kdc[9950](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 23:18:04 dist-test-slave-754n krb5kdc[9950](info): set up 2 sockets
Sep 23 23:18:04 dist-test-slave-754n krb5kdc[9950](info): commencing operation
krb5kdc: starting...
W20260923 23:18:05.567458 25487 mini_kdc.cc:121] Time spent starting KDC: real 0.997s user 0.001s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790205259943518-25487-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790205259943518-25487-0/krb5kdc/test-user.keytab.
Sep 23 23:18:05 dist-test-slave-754n krb5kdc[9950](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205485, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T23:18:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T23:18:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.227.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.227.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.227.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab.
Entry for principal kudu/127.24.227.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab.
I20260923 23:18:05.678092 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790205259943518-25487-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790205259943518-25487-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790205259943518-25487-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.254:38389
--webserver_interface=127.24.227.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab
--principal=kudu/127.24.227.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:45329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.227.254:38389
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:18:05.755952 9963 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:18:05.756140 9963 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:18:05.756165 9963 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:18:05.757244 9963 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 23:18:05.757278 9963 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:18:05.757292 9963 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:18:05.757304 9963 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 23:18:05.757316 9963 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 23:18:05.758689 9963 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:45329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790205259943518-25487-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790205259943518-25487-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.227.254:38389
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab
--principal=kudu/127.24.227.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.254:38389
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.227.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9963
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790205259943518-25487-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:18:05.758867 9963 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:18:05.759027 9963 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:18:05.764000 9968 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:18:05.764025 9963 server_base.cc:1034] running on GCE node
W20260923 23:18:05.763996 9971 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:18:05.763995 9969 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:18:05.764547 9963 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:18:05.764779 9963 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:18:05.765913 9963 hybrid_clock.cc:648] HybridClock initialized: now 1790205485765890 us; error 38 us; skew 500 ppm
Sep 23 23:18:05 dist-test-slave-754n krb5kdc[9950](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205485, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:18:05.768319 9963 init.cc:377] Logged in from keytab as kudu/127.24.227.254@KRBTEST.COM (short username kudu)
I20260923 23:18:05.768769 9963 webserver.cc:460] Webserver started at http://127.24.227.254:36199/ using document root <none> and password file <none>
I20260923 23:18:05.768994 9963 fs_manager.cc:359] Metadata directory not provided
I20260923 23:18:05.769037 9963 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:18:05.769181 9963 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:18:05.770171 9963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790205259943518-25487-0/minicluster-data/master-0/data/instance:
uuid: "3343154aaa3049288cb087d14f8df11a"
format_stamp: "Formatted at 2026-09-23 23:18:05 on dist-test-slave-754n"
server_key: "34338921313550ee0dc3d4f7aeb5a66d"
server_key_iv: "f0837a1bde8861ea92965670c8c8dc7e"
server_key_version: "encryptionkey@0"
I20260923 23:18:05.770525 9963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790205259943518-25487-0/minicluster-data/master-0/wal/instance:
uuid: "3343154aaa3049288cb087d14f8df11a"
format_stamp: "Formatted at 2026-09-23 23:18:05 on dist-test-slave-754n"
server_key: "34338921313550ee0dc3d4f7aeb5a66d"
server_key_iv: "f0837a1bde8861ea92965670c8c8dc7e"
server_key_version: "encryptionkey@0"
I20260923 23:18:05.771919 9963 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260923 23:18:05.772641 9978 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:18:05.772821 9963 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 23:18:05.772907 9963 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790205259943518-25487-0/minicluster-data/master-0/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790205259943518-25487-0/minicluster-data/master-0/wal
uuid: "3343154aaa3049288cb087d14f8df11a"
format_stamp: "Formatted at 2026-09-23 23:18:05 on dist-test-slave-754n"
server_key: "34338921313550ee0dc3d4f7aeb5a66d"
server_key_iv: "f0837a1bde8861ea92965670c8c8dc7e"
server_key_version: "encryptionkey@0"
I20260923 23:18:05.772980 9963 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790205259943518-25487-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790205259943518-25487-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790205259943518-25487-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:18:05.786382 9963 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:18:05.788194 9963 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:18:05.788339 9963 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:18:05.792114 9963 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.254:38389
I20260923 23:18:05.792168 10030 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.254:38389 every 8 connection(s)
I20260923 23:18:05.792515 9963 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
I20260923 23:18:05.793311 10031 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:18:05.795384 10031 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3343154aaa3049288cb087d14f8df11a: Bootstrap starting.
I20260923 23:18:05.795961 10031 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3343154aaa3049288cb087d14f8df11a: Neither blocks nor log segments found. Creating new log.
I20260923 23:18:05.796232 10031 log.cc:824] T 00000000000000000000000000000000 P 3343154aaa3049288cb087d14f8df11a: Log is configured to *not* fsync() on all Append() calls
I20260923 23:18:05.796854 10031 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3343154aaa3049288cb087d14f8df11a: No bootstrap required, opened a new log
I20260923 23:18:05.798367 10031 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3343154aaa3049288cb087d14f8df11a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3343154aaa3049288cb087d14f8df11a" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 38389 } }
I20260923 23:18:05.798484 10031 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3343154aaa3049288cb087d14f8df11a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:18:05.798504 10031 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3343154aaa3049288cb087d14f8df11a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3343154aaa3049288cb087d14f8df11a, State: Initialized, Role: FOLLOWER
I20260923 23:18:05.798662 10031 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3343154aaa3049288cb087d14f8df11a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3343154aaa3049288cb087d14f8df11a" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 38389 } }
I20260923 23:18:05.798717 10031 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3343154aaa3049288cb087d14f8df11a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 23:18:05.798768 10031 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3343154aaa3049288cb087d14f8df11a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 23:18:05.798812 10031 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3343154aaa3049288cb087d14f8df11a [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:18:05.799285 10031 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3343154aaa3049288cb087d14f8df11a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3343154aaa3049288cb087d14f8df11a" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 38389 } }
I20260923 23:18:05.799384 10031 leader_election.cc:302] T 00000000000000000000000000000000 P 3343154aaa3049288cb087d14f8df11a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3343154aaa3049288cb087d14f8df11a; no voters:
I20260923 23:18:05.799539 10031 leader_election.cc:288] T 00000000000000000000000000000000 P 3343154aaa3049288cb087d14f8df11a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 23:18:05.799566 10034 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3343154aaa3049288cb087d14f8df11a [term 1 FOLLOWER]: Leader election won for term 1
I20260923 23:18:05.799718 10034 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3343154aaa3049288cb087d14f8df11a [term 1 LEADER]: Becoming Leader. State: Replica: 3343154aaa3049288cb087d14f8df11a, State: Running, Role: LEADER
I20260923 23:18:05.799813 10031 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3343154aaa3049288cb087d14f8df11a [sys.catalog]: configured and running, proceeding with master startup.
I20260923 23:18:05.799904 10034 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3343154aaa3049288cb087d14f8df11a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3343154aaa3049288cb087d14f8df11a" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 38389 } }
I20260923 23:18:05.800494 10036 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3343154aaa3049288cb087d14f8df11a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3343154aaa3049288cb087d14f8df11a. Latest consensus state: current_term: 1 leader_uuid: "3343154aaa3049288cb087d14f8df11a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3343154aaa3049288cb087d14f8df11a" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 38389 } } }
I20260923 23:18:05.800592 10036 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3343154aaa3049288cb087d14f8df11a [sys.catalog]: This master's current role is: LEADER
I20260923 23:18:05.800748 10035 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3343154aaa3049288cb087d14f8df11a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3343154aaa3049288cb087d14f8df11a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3343154aaa3049288cb087d14f8df11a" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 38389 } } }
I20260923 23:18:05.800837 10035 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3343154aaa3049288cb087d14f8df11a [sys.catalog]: This master's current role is: LEADER
I20260923 23:18:05.800858 10043 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 23:18:05.801322 10043 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 23:18:05.802177 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 9963
I20260923 23:18:05.802299 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790205259943518-25487-0/minicluster-data/master-0/wal/instance
I20260923 23:18:05.802484 25487 external_mini_cluster.cc:1428] Setting key 1e19a30b1b1f7ac427e9fedd849f8c47
I20260923 23:18:05.803380 10043 catalog_manager.cc:1347] Generated new cluster ID: facc596317b348d7a27f065e27757e25
I20260923 23:18:05.803429 10043 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 23:18:05 dist-test-slave-754n krb5kdc[9950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205485, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
I20260923 23:18:05.812134 10043 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 23:18:05.812680 10043 catalog_manager.cc:1504] Loading token signing keys...
I20260923 23:18:05.813786 10054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:05.803100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44896 (local address 127.24.227.254:38389)
0923 23:18:05.803239 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:05.803248 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:05.803278 (+ 30us) server_negotiation.cc:407] Connection header received
0923 23:18:05.803385 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:05.803402 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:05.803453 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:05.803589 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:05.804280 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:05.805690 (+ 1410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:05.806735 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:05.807353 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:05.807478 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:05.810099 (+ 2621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:18:05.810102 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:18:05.810103 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:18:05.810113 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:18:05.812184 (+ 2071us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:05.812591 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:05.812594 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:05.812594 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:05.812692 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:05.812939 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:05.812940 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:05.812940 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:05.813174 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:18:05.813312 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:05.813458 (+ 146us) server_negotiation.cc:299] Negotiation successful
0923 23:18:05.813584 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":73,"threads_started":1}
I20260923 23:18:05.821379 10043 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3343154aaa3049288cb087d14f8df11a: Generated new TSK 0
I20260923 23:18:05.821537 10043 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.227.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.227.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.227.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.193.keytab.
Entry for principal kudu/127.24.227.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.193.keytab.
I20260923 23:18:05.854833 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790205259943518-25487-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790205259943518-25487-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790205259943518-25487-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790205259943518-25487-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.227.193:0
--local_ip_for_outbound_sockets=127.24.227.193
--webserver_interface=127.24.227.193
--webserver_port=0
--tserver_master_addrs=127.24.227.254:38389
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.193.keytab
--principal=kudu/127.24.227.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:45329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:18:05.934916 10057 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:18:05.935097 10057 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:18:05.935119 10057 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:18:05.936216 10057 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:18:05.936254 10057 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:18:05.936280 10057 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.227.193
I20260923 23:18:05.937655 10057 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:45329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790205259943518-25487-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790205259943518-25487-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.193.keytab
--principal=kudu/127.24.227.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.227.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790205259943518-25487-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.227.193
--webserver_port=0
--tserver_master_addrs=127.24.227.254:38389
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10057
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.227.193
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790205259943518-25487-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:18:05.937881 10057 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:18:05.938117 10057 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:18:05.942983 10062 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:18:05.943079 10057 server_base.cc:1034] running on GCE node
W20260923 23:18:05.943150 10065 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:18:05.942996 10063 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:18:05.943393 10057 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:18:05.943640 10057 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:18:05.944792 10057 hybrid_clock.cc:648] HybridClock initialized: now 1790205485944755 us; error 52 us; skew 500 ppm
Sep 23 23:18:05 dist-test-slave-754n krb5kdc[9950](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205485, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:18:05.947225 10057 init.cc:377] Logged in from keytab as kudu/127.24.227.193@KRBTEST.COM (short username kudu)
I20260923 23:18:05.947599 10057 webserver.cc:460] Webserver started at http://127.24.227.193:45435/ using document root <none> and password file <none>
I20260923 23:18:05.947829 10057 fs_manager.cc:359] Metadata directory not provided
I20260923 23:18:05.947921 10057 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:18:05.948074 10057 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:18:05.949018 10057 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790205259943518-25487-0/minicluster-data/ts-0/data/instance:
uuid: "824c15d92f1446e0a6e9c1484170e3a1"
format_stamp: "Formatted at 2026-09-23 23:18:05 on dist-test-slave-754n"
server_key: "ceb0ea320df12e7daa5f39d42dab5bf0"
server_key_iv: "abff5009f1e3467ef1eae98b789910aa"
server_key_version: "encryptionkey@0"
I20260923 23:18:05.949369 10057 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790205259943518-25487-0/minicluster-data/ts-0/wal/instance:
uuid: "824c15d92f1446e0a6e9c1484170e3a1"
format_stamp: "Formatted at 2026-09-23 23:18:05 on dist-test-slave-754n"
server_key: "ceb0ea320df12e7daa5f39d42dab5bf0"
server_key_iv: "abff5009f1e3467ef1eae98b789910aa"
server_key_version: "encryptionkey@0"
I20260923 23:18:05.950635 10057 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260923 23:18:05.951289 10072 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:18:05.951459 10057 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 23:18:05.951555 10057 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790205259943518-25487-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790205259943518-25487-0/minicluster-data/ts-0/wal
uuid: "824c15d92f1446e0a6e9c1484170e3a1"
format_stamp: "Formatted at 2026-09-23 23:18:05 on dist-test-slave-754n"
server_key: "ceb0ea320df12e7daa5f39d42dab5bf0"
server_key_iv: "abff5009f1e3467ef1eae98b789910aa"
server_key_version: "encryptionkey@0"
I20260923 23:18:05.951638 10057 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790205259943518-25487-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790205259943518-25487-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790205259943518-25487-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:18:05.989822 10057 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:18:05.992199 10057 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:18:05.992373 10057 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:18:05.992621 10057 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 23:18:05.992931 10057 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 23:18:05.992987 10057 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 23:18:05.993028 10057 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 23:18:05.993055 10057 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 23:18:05.998596 10057 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.193:41327
I20260923 23:18:05.998690 10185 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.193:41327 every 8 connection(s)
I20260923 23:18:05.998967 10057 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790205259943518-25487-0/minicluster-data/ts-0/data/info.pb
I20260923 23:18:05.999836 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 10057
I20260923 23:18:05.999991 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790205259943518-25487-0/minicluster-data/ts-0/wal/instance
I20260923 23:18:06.000206 25487 external_mini_cluster.cc:1428] Setting key e49ac01827db0457807513fe078171da
Sep 23 23:18:06 dist-test-slave-754n krb5kdc[9950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205485, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.193@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.24.227.194@KRBTEST.COM; defaulting to no policy
I20260923 23:18:06.009488 10054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:05.999765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.193:58339 (local address 127.24.227.254:38389)
0923 23:18:05.999833 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:05.999836 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:06.000023 (+ 187us) server_negotiation.cc:407] Connection header received
0923 23:18:06.000713 (+ 690us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:06.000714 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:06.000722 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:06.000803 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:06.002076 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:06.002776 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:06.004094 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:06.004291 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:06.004348 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:06.006595 (+ 2247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:18:06.006597 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:18:06.006597 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:18:06.006603 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:18:06.008160 (+ 1557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:06.008550 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:06.008552 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:06.008552 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:06.008609 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:06.008851 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:06.008852 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:06.008852 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:06.008956 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:18:06.009031 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:06.009278 (+ 247us) server_negotiation.cc:299] Negotiation successful
0923 23:18:06.009372 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260923 23:18:06.009593 10188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:05.999716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:38389 (local address 127.24.227.193:58339)
0923 23:18:05.999906 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:05.999946 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0923 23:18:06.000621 (+ 675us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:18:06.000906 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:18:06.000910 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:18:06.001716 (+ 806us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:18:06.001985 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:06.001992 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:06.002884 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:06.002885 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:06.004012 (+ 1127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:06.004014 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:06.004382 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:06.004383 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:06.004531 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:06.004985 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:18:06.004997 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:18:06.006505 (+ 1508us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:18:06.008253 (+ 1748us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:18:06.008260 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:18:06.008262 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:18:06.008475 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:18:06.008672 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:18:06.008673 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:18:06.008673 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:18:06.008780 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:18:06.009021 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:18:06.009028 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:18:06.009156 (+ 128us) client_negotiation.cc:769] Sending connection context
0923 23:18:06.009256 (+ 100us) client_negotiation.cc:240] Negotiation successful
0923 23:18:06.009393 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":84,"threads_started":1}
I20260923 23:18:06.010006 10186 heartbeater.cc:347] Connected to a master server at 127.24.227.254:38389
I20260923 23:18:06.010104 10186 heartbeater.cc:464] Registering TS with master...
I20260923 23:18:06.010250 10186 heartbeater.cc:511] Master 127.24.227.254:38389 requested a full tablet report, sending...
I20260923 23:18:06.011176 9995 ts_manager.cc:194] Registered new tserver with Master: 824c15d92f1446e0a6e9c1484170e3a1 (127.24.227.193:41327)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.227.194@KRBTEST.COM" created.
I20260923 23:18:06.011829 9995 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.227.193@KRBTEST.COM'} at 127.24.227.193:58339
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.227.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.194.keytab.
Entry for principal kudu/127.24.227.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.194.keytab.
I20260923 23:18:06.026737 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790205259943518-25487-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790205259943518-25487-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790205259943518-25487-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790205259943518-25487-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.227.194:0
--local_ip_for_outbound_sockets=127.24.227.194
--webserver_interface=127.24.227.194
--webserver_port=0
--tserver_master_addrs=127.24.227.254:38389
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.194.keytab
--principal=kudu/127.24.227.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:45329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:18:06.103348 10191 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:18:06.103585 10191 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:18:06.103623 10191 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:18:06.104764 10191 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:18:06.104830 10191 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:18:06.104874 10191 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.227.194
I20260923 23:18:06.106297 10191 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:45329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790205259943518-25487-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790205259943518-25487-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.194.keytab
--principal=kudu/127.24.227.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.227.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790205259943518-25487-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.227.194
--webserver_port=0
--tserver_master_addrs=127.24.227.254:38389
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10191
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.227.194
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790205259943518-25487-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:18:06.106546 10191 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:18:06.106770 10191 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:18:06.111539 10197 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:18:06.111539 10196 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:18:06.111527 10199 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:18:06.111724 10191 server_base.cc:1034] running on GCE node
I20260923 23:18:06.111963 10191 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:18:06.112218 10191 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:18:06.113384 10191 hybrid_clock.cc:648] HybridClock initialized: now 1790205486113357 us; error 39 us; skew 500 ppm
Sep 23 23:18:06 dist-test-slave-754n krb5kdc[9950](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205486, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:18:06.115787 10191 init.cc:377] Logged in from keytab as kudu/127.24.227.194@KRBTEST.COM (short username kudu)
I20260923 23:18:06.116218 10191 webserver.cc:460] Webserver started at http://127.24.227.194:40809/ using document root <none> and password file <none>
I20260923 23:18:06.116407 10191 fs_manager.cc:359] Metadata directory not provided
I20260923 23:18:06.116448 10191 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:18:06.116532 10191 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:18:06.117491 10191 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790205259943518-25487-0/minicluster-data/ts-1/data/instance:
uuid: "d20ee87894274b4aaa1b3f04353e9cf7"
format_stamp: "Formatted at 2026-09-23 23:18:06 on dist-test-slave-754n"
server_key: "60d6a52285d511b37191abb5cc905256"
server_key_iv: "58c224716daeba2c1b18c7f994d66473"
server_key_version: "encryptionkey@0"
I20260923 23:18:06.117813 10191 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790205259943518-25487-0/minicluster-data/ts-1/wal/instance:
uuid: "d20ee87894274b4aaa1b3f04353e9cf7"
format_stamp: "Formatted at 2026-09-23 23:18:06 on dist-test-slave-754n"
server_key: "60d6a52285d511b37191abb5cc905256"
server_key_iv: "58c224716daeba2c1b18c7f994d66473"
server_key_version: "encryptionkey@0"
I20260923 23:18:06.119005 10191 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 23:18:06.119663 10206 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:18:06.119812 10191 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 23:18:06.119920 10191 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790205259943518-25487-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790205259943518-25487-0/minicluster-data/ts-1/wal
uuid: "d20ee87894274b4aaa1b3f04353e9cf7"
format_stamp: "Formatted at 2026-09-23 23:18:06 on dist-test-slave-754n"
server_key: "60d6a52285d511b37191abb5cc905256"
server_key_iv: "58c224716daeba2c1b18c7f994d66473"
server_key_version: "encryptionkey@0"
I20260923 23:18:06.119990 10191 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790205259943518-25487-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790205259943518-25487-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790205259943518-25487-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:18:06.133985 10191 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:18:06.136219 10191 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:18:06.136322 10191 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:18:06.136487 10191 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 23:18:06.136711 10191 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 23:18:06.136739 10191 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 23:18:06.136757 10191 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 23:18:06.136770 10191 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 23:18:06.142016 10191 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.194:43659
I20260923 23:18:06.142076 10319 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.194:43659 every 8 connection(s)
I20260923 23:18:06.142346 10191 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790205259943518-25487-0/minicluster-data/ts-1/data/info.pb
Sep 23 23:18:06 dist-test-slave-754n krb5kdc[9950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205486, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.194@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
I20260923 23:18:06.150378 10054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:06.143045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.194:46115 (local address 127.24.227.254:38389)
0923 23:18:06.143116 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:06.143119 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:06.143227 (+ 108us) server_negotiation.cc:407] Connection header received
0923 23:18:06.143705 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:06.143707 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:06.143716 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:06.143760 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:06.144584 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:06.145085 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:06.146004 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:06.146176 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:06.146224 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:06.148057 (+ 1833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:18:06.148058 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:18:06.148058 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:18:06.148064 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:18:06.149351 (+ 1287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:06.149677 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:06.149677 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:06.149678 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:06.149722 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:06.149913 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:06.149914 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:06.149914 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:06.150003 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:18:06.150050 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:06.150236 (+ 186us) server_negotiation.cc:299] Negotiation successful
0923 23:18:06.150288 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260923 23:18:06.150592 10322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:06.142993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:38389 (local address 127.24.227.194:46115)
0923 23:18:06.143124 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:06.143154 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0923 23:18:06.143618 (+ 464us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:18:06.143816 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:18:06.143819 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:18:06.144344 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:18:06.144514 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:06.144518 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:06.145165 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:06.145166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:06.145938 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:06.145940 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:06.146243 (+ 303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:06.146244 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:06.146335 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:06.146694 (+ 359us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:18:06.146702 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:18:06.147983 (+ 1281us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:18:06.149422 (+ 1439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:18:06.149427 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:18:06.149428 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:18:06.149622 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:18:06.149775 (+ 153us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:18:06.149776 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:18:06.149776 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:18:06.149865 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:18:06.150053 (+ 188us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:18:06.150057 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:18:06.150154 (+ 97us) client_negotiation.cc:769] Sending connection context
0923 23:18:06.150226 (+ 72us) client_negotiation.cc:240] Negotiation successful
0923 23:18:06.150357 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":76,"threads_started":1}
I20260923 23:18:06.150890 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 10191
I20260923 23:18:06.151003 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790205259943518-25487-0/minicluster-data/ts-1/wal/instance
I20260923 23:18:06.151198 25487 external_mini_cluster.cc:1428] Setting key 4afc8f08afff3b995bbb819fe6ba787c
I20260923 23:18:06.151186 10320 heartbeater.cc:347] Connected to a master server at 127.24.227.254:38389
I20260923 23:18:06.151288 10320 heartbeater.cc:464] Registering TS with master...
I20260923 23:18:06.151477 10320 heartbeater.cc:511] Master 127.24.227.254:38389 requested a full tablet report, sending...
I20260923 23:18:06.151883 9995 ts_manager.cc:194] Registered new tserver with Master: d20ee87894274b4aaa1b3f04353e9cf7 (127.24.227.194:43659)
I20260923 23:18:06.152242 9995 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.227.194@KRBTEST.COM'} at 127.24.227.194:46115
WARNING: no policy specified for kudu/127.24.227.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.227.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.227.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.195.keytab.
Entry for principal kudu/127.24.227.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.195.keytab.
I20260923 23:18:06.174443 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790205259943518-25487-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790205259943518-25487-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790205259943518-25487-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790205259943518-25487-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.227.195:0
--local_ip_for_outbound_sockets=127.24.227.195
--webserver_interface=127.24.227.195
--webserver_port=0
--tserver_master_addrs=127.24.227.254:38389
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.195.keytab
--principal=kudu/127.24.227.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:45329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:18:06.251135 10325 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:18:06.251360 10325 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:18:06.251399 10325 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:18:06.252575 10325 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:18:06.252635 10325 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:18:06.252679 10325 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.227.195
I20260923 23:18:06.254115 10325 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:45329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790205259943518-25487-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790205259943518-25487-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.195.keytab
--principal=kudu/127.24.227.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.227.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790205259943518-25487-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.227.195
--webserver_port=0
--tserver_master_addrs=127.24.227.254:38389
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10325
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.227.195
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790205259943518-25487-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:18:06.254388 10325 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:18:06.254616 10325 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:18:06.259163 10330 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:18:06.259198 10325 server_base.cc:1034] running on GCE node
W20260923 23:18:06.259204 10331 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:18:06.259290 10333 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:18:06.259537 10325 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:18:06.259791 10325 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:18:06.260970 10325 hybrid_clock.cc:648] HybridClock initialized: now 1790205486260937 us; error 28 us; skew 500 ppm
Sep 23 23:18:06 dist-test-slave-754n krb5kdc[9950](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205486, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:18:06.263337 10325 init.cc:377] Logged in from keytab as kudu/127.24.227.195@KRBTEST.COM (short username kudu)
I20260923 23:18:06.263679 10325 webserver.cc:460] Webserver started at http://127.24.227.195:37795/ using document root <none> and password file <none>
I20260923 23:18:06.263945 10325 fs_manager.cc:359] Metadata directory not provided
I20260923 23:18:06.263990 10325 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:18:06.264127 10325 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:18:06.265074 10325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790205259943518-25487-0/minicluster-data/ts-2/data/instance:
uuid: "a4769a0ae42f4a6d90dd30080328cb6c"
format_stamp: "Formatted at 2026-09-23 23:18:06 on dist-test-slave-754n"
server_key: "45f5039dc740eb418437d77d9bb7d19b"
server_key_iv: "8e1c7408ec4db099a78a848fcefe1fdb"
server_key_version: "encryptionkey@0"
I20260923 23:18:06.265409 10325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790205259943518-25487-0/minicluster-data/ts-2/wal/instance:
uuid: "a4769a0ae42f4a6d90dd30080328cb6c"
format_stamp: "Formatted at 2026-09-23 23:18:06 on dist-test-slave-754n"
server_key: "45f5039dc740eb418437d77d9bb7d19b"
server_key_iv: "8e1c7408ec4db099a78a848fcefe1fdb"
server_key_version: "encryptionkey@0"
I20260923 23:18:06.266647 10325 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 23:18:06.267270 10340 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:18:06.267441 10325 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 23:18:06.267505 10325 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790205259943518-25487-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790205259943518-25487-0/minicluster-data/ts-2/wal
uuid: "a4769a0ae42f4a6d90dd30080328cb6c"
format_stamp: "Formatted at 2026-09-23 23:18:06 on dist-test-slave-754n"
server_key: "45f5039dc740eb418437d77d9bb7d19b"
server_key_iv: "8e1c7408ec4db099a78a848fcefe1fdb"
server_key_version: "encryptionkey@0"
I20260923 23:18:06.267584 10325 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790205259943518-25487-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790205259943518-25487-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790205259943518-25487-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:18:06.277568 10325 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:18:06.279649 10325 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:18:06.279804 10325 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:18:06.280045 10325 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 23:18:06.280330 10325 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 23:18:06.280385 10325 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 23:18:06.280423 10325 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 23:18:06.280452 10325 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 23:18:06.286063 10325 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.195:45575
I20260923 23:18:06.286134 10453 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.195:45575 every 8 connection(s)
I20260923 23:18:06.286427 10325 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790205259943518-25487-0/minicluster-data/ts-2/data/info.pb
I20260923 23:18:06.288475 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 10325
I20260923 23:18:06.288609 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790205259943518-25487-0/minicluster-data/ts-2/wal/instance
I20260923 23:18:06.288818 25487 external_mini_cluster.cc:1428] Setting key 6fdf29b7ed6ac16bae1dfd57b19dfbb1
Sep 23 23:18:06 dist-test-slave-754n krb5kdc[9950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205486, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.195@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
I20260923 23:18:06.295347 10054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:06.287088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.195:38609 (local address 127.24.227.254:38389)
0923 23:18:06.287152 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:06.287154 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:06.287397 (+ 243us) server_negotiation.cc:407] Connection header received
0923 23:18:06.287817 (+ 420us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:06.287818 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:06.287828 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:06.287955 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:06.288960 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:06.289680 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:06.290649 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:06.290827 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:06.290883 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:06.292970 (+ 2087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:18:06.292971 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:18:06.292971 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:18:06.292977 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:18:06.294138 (+ 1161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:06.294520 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:06.294521 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:06.294521 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:06.294572 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:06.294774 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:06.294775 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:06.294775 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:06.294892 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:18:06.294943 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:06.295229 (+ 286us) server_negotiation.cc:299] Negotiation successful
0923 23:18:06.295285 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260923 23:18:06.295583 10456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:06.287057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:38389 (local address 127.24.227.195:38609)
0923 23:18:06.287237 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:06.287281 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0923 23:18:06.287734 (+ 453us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:18:06.288027 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:18:06.288031 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:18:06.288552 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:18:06.288853 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:06.288858 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:06.289796 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:06.289797 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:06.290575 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:06.290577 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:06.290908 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:06.290909 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:06.291005 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:06.291552 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:18:06.291560 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:18:06.292892 (+ 1332us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:18:06.294214 (+ 1322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:18:06.294219 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:18:06.294221 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:18:06.294452 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:18:06.294631 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:18:06.294632 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:18:06.294632 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:18:06.294720 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:18:06.294989 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:18:06.294994 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:18:06.295104 (+ 110us) client_negotiation.cc:769] Sending connection context
0923 23:18:06.295203 (+ 99us) client_negotiation.cc:240] Negotiation successful
0923 23:18:06.295315 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":90,"threads_started":1}
I20260923 23:18:06.296082 10454 heartbeater.cc:347] Connected to a master server at 127.24.227.254:38389
I20260923 23:18:06.296161 10454 heartbeater.cc:464] Registering TS with master...
I20260923 23:18:06.296346 10454 heartbeater.cc:511] Master 127.24.227.254:38389 requested a full tablet report, sending...
I20260923 23:18:06.296717 9995 ts_manager.cc:194] Registered new tserver with Master: a4769a0ae42f4a6d90dd30080328cb6c (127.24.227.195:45575)
I20260923 23:18:06.297076 9995 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.227.195@KRBTEST.COM'} at 127.24.227.195:38609
I20260923 23:18:06.300561 25487 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 23:18:06.307287 10054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:06.302026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44900 (local address 127.24.227.254:38389)
0923 23:18:06.302068 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:06.302070 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:06.302171 (+ 101us) server_negotiation.cc:407] Connection header received
0923 23:18:06.302219 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:06.302220 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:06.302227 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:06.302278 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:06.302781 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:06.303215 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:06.303930 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:06.304109 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:06.304158 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:06.305276 (+ 1118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:18:06.305277 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:18:06.305277 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:18:06.305282 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:18:06.306418 (+ 1136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:06.306713 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:06.306713 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:06.306714 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:06.306759 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:06.306912 (+ 153us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:06.306912 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:06.306912 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:06.307030 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:18:06.307105 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:06.307180 (+ 75us) server_negotiation.cc:299] Negotiation successful
0923 23:18:06.307232 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260923 23:18:06.308709 9995 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44900:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 23:18:06.310142 9995 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 23:18:06.316469 10464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:06.312251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.194:43659 (local address 127.0.0.1:46986)
0923 23:18:06.312599 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:06.312609 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 23:18:06.312645 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:18:06.313042 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:18:06.313042 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:18:06.313044 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:18:06.313079 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:06.313081 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:06.314100 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:06.314101 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:06.315373 (+ 1272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:06.315375 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:06.316187 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:06.316188 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:06.316266 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:06.316278 (+ 12us) client_negotiation.cc:769] Sending connection context
0923 23:18:06.316346 (+ 68us) client_negotiation.cc:240] Negotiation successful
0923 23:18:06.316399 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"mutex_wait_us":5,"thread_start_us":50,"threads_started":1}
I20260923 23:18:06.316524 10466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:06.312443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46986 (local address 127.24.227.194:43659)
0923 23:18:06.312883 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:06.312888 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:06.312899 (+ 11us) server_negotiation.cc:407] Connection header received
0923 23:18:06.312939 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:06.312941 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:06.312957 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:06.313031 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:18:06.313175 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:06.314000 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:06.315464 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:06.316103 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:06.316187 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:06.316240 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:06.316372 (+ 132us) server_negotiation.cc:299] Negotiation successful
0923 23:18:06.316412 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":88,"threads_started":1}
I20260923 23:18:06.316792 10462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:06.312261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.195:45575 (local address 127.0.0.1:37972)
0923 23:18:06.313126 (+ 865us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:06.313136 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 23:18:06.313220 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:18:06.313428 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:18:06.313429 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:18:06.313431 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:18:06.313477 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:06.313478 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:06.314676 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:06.314677 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:06.315633 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:06.315636 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:06.316536 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:06.316536 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:06.316636 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:06.316641 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 23:18:06.316701 (+ 60us) client_negotiation.cc:240] Negotiation successful
0923 23:18:06.316734 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":851,"mutex_wait_us":2,"thread_start_us":72,"threads_started":1}
I20260923 23:18:06.316793 10467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:06.312653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45760 (local address 127.24.227.193:41327)
0923 23:18:06.312790 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:06.312793 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:06.312801 (+ 8us) server_negotiation.cc:407] Connection header received
0923 23:18:06.312833 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:06.312834 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:06.312846 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:06.312941 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:18:06.313105 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:06.314213 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:06.315497 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:06.316220 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:06.316290 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:06.316337 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:06.316667 (+ 330us) server_negotiation.cc:299] Negotiation successful
0923 23:18:06.316716 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":79,"threads_started":1}
I20260923 23:18:06.316845 10465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:06.312445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37972 (local address 127.24.227.195:45575)
0923 23:18:06.312603 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:06.312607 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:06.313190 (+ 583us) server_negotiation.cc:407] Connection header received
0923 23:18:06.313310 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:06.313313 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:06.313327 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:06.313414 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:18:06.313534 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:06.314577 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:06.315722 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:06.316399 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:06.316460 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:06.316506 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:06.316728 (+ 222us) server_negotiation.cc:299] Negotiation successful
0923 23:18:06.316767 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":99,"threads_started":1}
I20260923 23:18:06.316602 10463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:06.312251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.193:41327 (local address 127.0.0.1:45760)
0923 23:18:06.312402 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:06.312440 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0923 23:18:06.312504 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:18:06.312949 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:18:06.312952 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:18:06.312954 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:18:06.313021 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:06.313023 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:06.314320 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:06.314321 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:06.315413 (+ 1092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:06.315415 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:06.316372 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:06.316373 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:06.316438 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:06.316443 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 23:18:06.316487 (+ 44us) client_negotiation.cc:240] Negotiation successful
0923 23:18:06.316530 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"mutex_wait_us":5,"thread_start_us":69,"threads_started":1}
I20260923 23:18:06.317524 10254 tablet_service.cc:1467] Processing CreateTablet for tablet efa463299829459c8b8aba3ebde7a538 (DEFAULT_TABLE table=test-table [id=9b9ce3a73d094450b4b97d4eaa287c0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 23:18:06.317883 10254 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet efa463299829459c8b8aba3ebde7a538. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:18:06.317821 10388 tablet_service.cc:1467] Processing CreateTablet for tablet efa463299829459c8b8aba3ebde7a538 (DEFAULT_TABLE table=test-table [id=9b9ce3a73d094450b4b97d4eaa287c0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 23:18:06.318002 10120 tablet_service.cc:1467] Processing CreateTablet for tablet efa463299829459c8b8aba3ebde7a538 (DEFAULT_TABLE table=test-table [id=9b9ce3a73d094450b4b97d4eaa287c0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 23:18:06.318114 10388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet efa463299829459c8b8aba3ebde7a538. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:18:06.318230 10120 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet efa463299829459c8b8aba3ebde7a538. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:18:06.320683 10468 tablet_bootstrap.cc:492] T efa463299829459c8b8aba3ebde7a538 P 824c15d92f1446e0a6e9c1484170e3a1: Bootstrap starting.
I20260923 23:18:06.321326 10469 tablet_bootstrap.cc:492] T efa463299829459c8b8aba3ebde7a538 P d20ee87894274b4aaa1b3f04353e9cf7: Bootstrap starting.
I20260923 23:18:06.321338 10470 tablet_bootstrap.cc:492] T efa463299829459c8b8aba3ebde7a538 P a4769a0ae42f4a6d90dd30080328cb6c: Bootstrap starting.
I20260923 23:18:06.321376 10468 tablet_bootstrap.cc:654] T efa463299829459c8b8aba3ebde7a538 P 824c15d92f1446e0a6e9c1484170e3a1: Neither blocks nor log segments found. Creating new log.
I20260923 23:18:06.321973 10469 tablet_bootstrap.cc:654] T efa463299829459c8b8aba3ebde7a538 P d20ee87894274b4aaa1b3f04353e9cf7: Neither blocks nor log segments found. Creating new log.
I20260923 23:18:06.321973 10470 tablet_bootstrap.cc:654] T efa463299829459c8b8aba3ebde7a538 P a4769a0ae42f4a6d90dd30080328cb6c: Neither blocks nor log segments found. Creating new log.
I20260923 23:18:06.321961 10468 log.cc:824] T efa463299829459c8b8aba3ebde7a538 P 824c15d92f1446e0a6e9c1484170e3a1: Log is configured to *not* fsync() on all Append() calls
I20260923 23:18:06.322593 10470 log.cc:824] T efa463299829459c8b8aba3ebde7a538 P a4769a0ae42f4a6d90dd30080328cb6c: Log is configured to *not* fsync() on all Append() calls
I20260923 23:18:06.322593 10469 log.cc:824] T efa463299829459c8b8aba3ebde7a538 P d20ee87894274b4aaa1b3f04353e9cf7: Log is configured to *not* fsync() on all Append() calls
I20260923 23:18:06.322710 10468 tablet_bootstrap.cc:492] T efa463299829459c8b8aba3ebde7a538 P 824c15d92f1446e0a6e9c1484170e3a1: No bootstrap required, opened a new log
I20260923 23:18:06.322778 10468 ts_tablet_manager.cc:1397] T efa463299829459c8b8aba3ebde7a538 P 824c15d92f1446e0a6e9c1484170e3a1: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260923 23:18:06.323369 10470 tablet_bootstrap.cc:492] T efa463299829459c8b8aba3ebde7a538 P a4769a0ae42f4a6d90dd30080328cb6c: No bootstrap required, opened a new log
I20260923 23:18:06.323473 10470 ts_tablet_manager.cc:1397] T efa463299829459c8b8aba3ebde7a538 P a4769a0ae42f4a6d90dd30080328cb6c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260923 23:18:06.323556 10469 tablet_bootstrap.cc:492] T efa463299829459c8b8aba3ebde7a538 P d20ee87894274b4aaa1b3f04353e9cf7: No bootstrap required, opened a new log
I20260923 23:18:06.323622 10469 ts_tablet_manager.cc:1397] T efa463299829459c8b8aba3ebde7a538 P d20ee87894274b4aaa1b3f04353e9cf7: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260923 23:18:06.324760 10468 raft_consensus.cc:348] T efa463299829459c8b8aba3ebde7a538 P 824c15d92f1446e0a6e9c1484170e3a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "824c15d92f1446e0a6e9c1484170e3a1" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 41327 } } peers { permanent_uuid: "d20ee87894274b4aaa1b3f04353e9cf7" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 43659 } } peers { permanent_uuid: "a4769a0ae42f4a6d90dd30080328cb6c" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 45575 } }
I20260923 23:18:06.324913 10468 raft_consensus.cc:374] T efa463299829459c8b8aba3ebde7a538 P 824c15d92f1446e0a6e9c1484170e3a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:18:06.324973 10468 raft_consensus.cc:729] T efa463299829459c8b8aba3ebde7a538 P 824c15d92f1446e0a6e9c1484170e3a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 824c15d92f1446e0a6e9c1484170e3a1, State: Initialized, Role: FOLLOWER
I20260923 23:18:06.325089 10468 consensus_queue.cc:260] T efa463299829459c8b8aba3ebde7a538 P 824c15d92f1446e0a6e9c1484170e3a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "824c15d92f1446e0a6e9c1484170e3a1" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 41327 } } peers { permanent_uuid: "d20ee87894274b4aaa1b3f04353e9cf7" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 43659 } } peers { permanent_uuid: "a4769a0ae42f4a6d90dd30080328cb6c" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 45575 } }
I20260923 23:18:06.325348 10468 ts_tablet_manager.cc:1428] T efa463299829459c8b8aba3ebde7a538 P 824c15d92f1446e0a6e9c1484170e3a1: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 23:18:06.325480 10186 heartbeater.cc:503] Master 127.24.227.254:38389 was elected leader, sending a full tablet report...
I20260923 23:18:06.325434 10470 raft_consensus.cc:348] T efa463299829459c8b8aba3ebde7a538 P a4769a0ae42f4a6d90dd30080328cb6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "824c15d92f1446e0a6e9c1484170e3a1" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 41327 } } peers { permanent_uuid: "d20ee87894274b4aaa1b3f04353e9cf7" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 43659 } } peers { permanent_uuid: "a4769a0ae42f4a6d90dd30080328cb6c" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 45575 } }
I20260923 23:18:06.325485 10469 raft_consensus.cc:348] T efa463299829459c8b8aba3ebde7a538 P d20ee87894274b4aaa1b3f04353e9cf7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "824c15d92f1446e0a6e9c1484170e3a1" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 41327 } } peers { permanent_uuid: "d20ee87894274b4aaa1b3f04353e9cf7" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 43659 } } peers { permanent_uuid: "a4769a0ae42f4a6d90dd30080328cb6c" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 45575 } }
I20260923 23:18:06.325613 10469 raft_consensus.cc:374] T efa463299829459c8b8aba3ebde7a538 P d20ee87894274b4aaa1b3f04353e9cf7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:18:06.325608 10470 raft_consensus.cc:374] T efa463299829459c8b8aba3ebde7a538 P a4769a0ae42f4a6d90dd30080328cb6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:18:06.325650 10470 raft_consensus.cc:729] T efa463299829459c8b8aba3ebde7a538 P a4769a0ae42f4a6d90dd30080328cb6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4769a0ae42f4a6d90dd30080328cb6c, State: Initialized, Role: FOLLOWER
I20260923 23:18:06.325688 10469 raft_consensus.cc:729] T efa463299829459c8b8aba3ebde7a538 P d20ee87894274b4aaa1b3f04353e9cf7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d20ee87894274b4aaa1b3f04353e9cf7, State: Initialized, Role: FOLLOWER
I20260923 23:18:06.325786 10470 consensus_queue.cc:260] T efa463299829459c8b8aba3ebde7a538 P a4769a0ae42f4a6d90dd30080328cb6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "824c15d92f1446e0a6e9c1484170e3a1" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 41327 } } peers { permanent_uuid: "d20ee87894274b4aaa1b3f04353e9cf7" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 43659 } } peers { permanent_uuid: "a4769a0ae42f4a6d90dd30080328cb6c" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 45575 } }
I20260923 23:18:06.325789 10469 consensus_queue.cc:260] T efa463299829459c8b8aba3ebde7a538 P d20ee87894274b4aaa1b3f04353e9cf7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "824c15d92f1446e0a6e9c1484170e3a1" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 41327 } } peers { permanent_uuid: "d20ee87894274b4aaa1b3f04353e9cf7" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 43659 } } peers { permanent_uuid: "a4769a0ae42f4a6d90dd30080328cb6c" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 45575 } }
I20260923 23:18:06.326045 10470 ts_tablet_manager.cc:1428] T efa463299829459c8b8aba3ebde7a538 P a4769a0ae42f4a6d90dd30080328cb6c: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260923 23:18:06.326045 10469 ts_tablet_manager.cc:1428] T efa463299829459c8b8aba3ebde7a538 P d20ee87894274b4aaa1b3f04353e9cf7: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260923 23:18:06.326151 10320 heartbeater.cc:503] Master 127.24.227.254:38389 was elected leader, sending a full tablet report...
I20260923 23:18:06.326151 10454 heartbeater.cc:503] Master 127.24.227.254:38389 was elected leader, sending a full tablet report...
W20260923 23:18:06.392896 10321 tablet.cc:2369] T efa463299829459c8b8aba3ebde7a538 P d20ee87894274b4aaa1b3f04353e9cf7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 23:18:06.406510 10474 raft_consensus.cc:482] T efa463299829459c8b8aba3ebde7a538 P 824c15d92f1446e0a6e9c1484170e3a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 23:18:06.406611 10474 raft_consensus.cc:504] T efa463299829459c8b8aba3ebde7a538 P 824c15d92f1446e0a6e9c1484170e3a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "824c15d92f1446e0a6e9c1484170e3a1" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 41327 } } peers { permanent_uuid: "d20ee87894274b4aaa1b3f04353e9cf7" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 43659 } } peers { permanent_uuid: "a4769a0ae42f4a6d90dd30080328cb6c" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 45575 } }
I20260923 23:18:06.406946 10474 leader_election.cc:288] T efa463299829459c8b8aba3ebde7a538 P 824c15d92f1446e0a6e9c1484170e3a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d20ee87894274b4aaa1b3f04353e9cf7 (127.24.227.194:43659), a4769a0ae42f4a6d90dd30080328cb6c (127.24.227.195:45575)
I20260923 23:18:06.409479 10466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:06.407157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.193:48665 (local address 127.24.227.194:43659)
0923 23:18:06.407199 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:06.407201 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:06.407211 (+ 10us) server_negotiation.cc:407] Connection header received
0923 23:18:06.407260 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:06.407262 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:06.407271 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:06.407325 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:18:06.407487 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:06.407979 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:06.408790 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:06.409131 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:06.409230 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:06.409238 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:06.409374 (+ 136us) server_negotiation.cc:299] Negotiation successful
0923 23:18:06.409412 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 23:18:06.409480 10188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:06.407076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.194:43659 (local address 127.24.227.193:48665)
0923 23:18:06.407149 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:06.407162 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 23:18:06.407205 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:18:06.407329 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:18:06.407330 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:18:06.407332 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:18:06.407414 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:06.407416 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:06.408050 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:06.408051 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:06.408707 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:06.408708 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:06.409226 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:06.409227 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:06.409302 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:06.409305 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 23:18:06.409375 (+ 70us) client_negotiation.cc:240] Negotiation successful
0923 23:18:06.409413 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260923 23:18:06.409551 10465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:06.407226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.193:56515 (local address 127.24.227.195:45575)
0923 23:18:06.407262 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:06.407263 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:06.407346 (+ 83us) server_negotiation.cc:407] Connection header received
0923 23:18:06.407390 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:06.407391 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:06.407400 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:06.407464 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:18:06.407607 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:06.408041 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:06.408867 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:06.409219 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:06.409299 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:06.409314 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:06.409447 (+ 133us) server_negotiation.cc:299] Negotiation successful
0923 23:18:06.409482 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260923 23:18:06.409607 10477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:06.407145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.195:45575 (local address 127.24.227.193:56515)
0923 23:18:06.407297 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:06.407305 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 23:18:06.407344 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:18:06.407466 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:18:06.407467 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:18:06.407468 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:18:06.407542 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:06.407543 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:06.408123 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:06.408124 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:06.408776 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:06.408778 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:06.409299 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:06.409300 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:06.409373 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:06.409376 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 23:18:06.409442 (+ 66us) client_negotiation.cc:240] Negotiation successful
0923 23:18:06.409477 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":113,"threads_started":1}
I20260923 23:18:06.409837 10274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efa463299829459c8b8aba3ebde7a538" candidate_uuid: "824c15d92f1446e0a6e9c1484170e3a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d20ee87894274b4aaa1b3f04353e9cf7" is_pre_election: true
I20260923 23:18:06.409946 10274 raft_consensus.cc:2454] T efa463299829459c8b8aba3ebde7a538 P d20ee87894274b4aaa1b3f04353e9cf7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 824c15d92f1446e0a6e9c1484170e3a1 in term 0.
I20260923 23:18:06.410106 10408 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efa463299829459c8b8aba3ebde7a538" candidate_uuid: "824c15d92f1446e0a6e9c1484170e3a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4769a0ae42f4a6d90dd30080328cb6c" is_pre_election: true
I20260923 23:18:06.410192 10073 leader_election.cc:302] T efa463299829459c8b8aba3ebde7a538 P 824c15d92f1446e0a6e9c1484170e3a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 824c15d92f1446e0a6e9c1484170e3a1, d20ee87894274b4aaa1b3f04353e9cf7; no voters:
I20260923 23:18:06.410248 10408 raft_consensus.cc:2454] T efa463299829459c8b8aba3ebde7a538 P a4769a0ae42f4a6d90dd30080328cb6c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 824c15d92f1446e0a6e9c1484170e3a1 in term 0.
I20260923 23:18:06.410326 10474 raft_consensus.cc:2781] T efa463299829459c8b8aba3ebde7a538 P 824c15d92f1446e0a6e9c1484170e3a1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 23:18:06.410427 10474 raft_consensus.cc:482] T efa463299829459c8b8aba3ebde7a538 P 824c15d92f1446e0a6e9c1484170e3a1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 23:18:06.410457 10474 raft_consensus.cc:3037] T efa463299829459c8b8aba3ebde7a538 P 824c15d92f1446e0a6e9c1484170e3a1 [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:18:06.411130 10474 raft_consensus.cc:504] T efa463299829459c8b8aba3ebde7a538 P 824c15d92f1446e0a6e9c1484170e3a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "824c15d92f1446e0a6e9c1484170e3a1" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 41327 } } peers { permanent_uuid: "d20ee87894274b4aaa1b3f04353e9cf7" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 43659 } } peers { permanent_uuid: "a4769a0ae42f4a6d90dd30080328cb6c" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 45575 } }
I20260923 23:18:06.411270 10474 leader_election.cc:288] T efa463299829459c8b8aba3ebde7a538 P 824c15d92f1446e0a6e9c1484170e3a1 [CANDIDATE]: Term 1 election: Requested vote from peers d20ee87894274b4aaa1b3f04353e9cf7 (127.24.227.194:43659), a4769a0ae42f4a6d90dd30080328cb6c (127.24.227.195:45575)
I20260923 23:18:06.411438 10408 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efa463299829459c8b8aba3ebde7a538" candidate_uuid: "824c15d92f1446e0a6e9c1484170e3a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4769a0ae42f4a6d90dd30080328cb6c"
I20260923 23:18:06.411473 10274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efa463299829459c8b8aba3ebde7a538" candidate_uuid: "824c15d92f1446e0a6e9c1484170e3a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d20ee87894274b4aaa1b3f04353e9cf7"
I20260923 23:18:06.411523 10408 raft_consensus.cc:3037] T efa463299829459c8b8aba3ebde7a538 P a4769a0ae42f4a6d90dd30080328cb6c [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:18:06.411528 10274 raft_consensus.cc:3037] T efa463299829459c8b8aba3ebde7a538 P d20ee87894274b4aaa1b3f04353e9cf7 [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:18:06.412297 10274 raft_consensus.cc:2454] T efa463299829459c8b8aba3ebde7a538 P d20ee87894274b4aaa1b3f04353e9cf7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 824c15d92f1446e0a6e9c1484170e3a1 in term 1.
I20260923 23:18:06.412369 10408 raft_consensus.cc:2454] T efa463299829459c8b8aba3ebde7a538 P a4769a0ae42f4a6d90dd30080328cb6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 824c15d92f1446e0a6e9c1484170e3a1 in term 1.
I20260923 23:18:06.412480 10073 leader_election.cc:302] T efa463299829459c8b8aba3ebde7a538 P 824c15d92f1446e0a6e9c1484170e3a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 824c15d92f1446e0a6e9c1484170e3a1, d20ee87894274b4aaa1b3f04353e9cf7; no voters:
I20260923 23:18:06.412604 10474 raft_consensus.cc:2781] T efa463299829459c8b8aba3ebde7a538 P 824c15d92f1446e0a6e9c1484170e3a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 23:18:06.412725 10474 raft_consensus.cc:686] T efa463299829459c8b8aba3ebde7a538 P 824c15d92f1446e0a6e9c1484170e3a1 [term 1 LEADER]: Becoming Leader. State: Replica: 824c15d92f1446e0a6e9c1484170e3a1, State: Running, Role: LEADER
I20260923 23:18:06.412819 10474 consensus_queue.cc:237] T efa463299829459c8b8aba3ebde7a538 P 824c15d92f1446e0a6e9c1484170e3a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "824c15d92f1446e0a6e9c1484170e3a1" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 41327 } } peers { permanent_uuid: "d20ee87894274b4aaa1b3f04353e9cf7" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 43659 } } peers { permanent_uuid: "a4769a0ae42f4a6d90dd30080328cb6c" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 45575 } }
I20260923 23:18:06.413539 9995 catalog_manager.cc:5495] T efa463299829459c8b8aba3ebde7a538 P 824c15d92f1446e0a6e9c1484170e3a1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 824c15d92f1446e0a6e9c1484170e3a1 (127.24.227.193). New cstate: current_term: 1 leader_uuid: "824c15d92f1446e0a6e9c1484170e3a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "824c15d92f1446e0a6e9c1484170e3a1" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 41327 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d20ee87894274b4aaa1b3f04353e9cf7" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 43659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4769a0ae42f4a6d90dd30080328cb6c" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 45575 } health_report { overall_health: UNKNOWN } } }
I20260923 23:18:06.430548 10467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:06.428578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45764 (local address 127.24.227.193:41327)
0923 23:18:06.428626 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:06.428628 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:06.428636 (+ 8us) server_negotiation.cc:407] Connection header received
0923 23:18:06.428675 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:06.428676 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:06.428689 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:06.428740 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 23:18:06.428859 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:06.429270 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:06.429877 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:06.430050 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:06.430097 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:06.430221 (+ 124us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 23:18:06.430313 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 23:18:06.430390 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:06.430432 (+ 42us) server_negotiation.cc:299] Negotiation successful
0923 23:18:06.430447 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260923 23:18:06.432019 10274 raft_consensus.cc:1261] T efa463299829459c8b8aba3ebde7a538 P d20ee87894274b4aaa1b3f04353e9cf7 [term 1 FOLLOWER]: Refusing update from remote peer 824c15d92f1446e0a6e9c1484170e3a1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 23:18:06.432093 10408 raft_consensus.cc:1261] T efa463299829459c8b8aba3ebde7a538 P a4769a0ae42f4a6d90dd30080328cb6c [term 1 FOLLOWER]: Refusing update from remote peer 824c15d92f1446e0a6e9c1484170e3a1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 23:18:06.432339 10474 consensus_queue.cc:1035] T efa463299829459c8b8aba3ebde7a538 P 824c15d92f1446e0a6e9c1484170e3a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d20ee87894274b4aaa1b3f04353e9cf7" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 23:18:06.432488 10478 consensus_queue.cc:1035] T efa463299829459c8b8aba3ebde7a538 P 824c15d92f1446e0a6e9c1484170e3a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4769a0ae42f4a6d90dd30080328cb6c" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 45575 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 23:18:06.434695 10481 mvcc.cc:205] Tried to move back new op lower bound from 7332681672423378944 to 7332681672347402240. Current Snapshot: MvccSnapshot[applied={T|T < 7332681672423378944}]
I20260923 23:18:06.435042 10483 mvcc.cc:205] Tried to move back new op lower bound from 7332681672423378944 to 7332681672347402240. Current Snapshot: MvccSnapshot[applied={T|T < 7332681672423378944}]
I20260923 23:18:06.435245 10482 mvcc.cc:205] Tried to move back new op lower bound from 7332681672423378944 to 7332681672347402240. Current Snapshot: MvccSnapshot[applied={T|T < 7332681672423378944}]
I20260923 23:18:06.437094 9994 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44900:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 23:18:06.437186 9994 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44900:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 23:18:06.437913 9994 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3343154aaa3049288cb087d14f8df11a: Sending DeleteTablet for 3 replicas of tablet efa463299829459c8b8aba3ebde7a538
I20260923 23:18:06.438220 10254 tablet_service.cc:1514] Processing DeleteTablet for tablet efa463299829459c8b8aba3ebde7a538 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 23:18:06 UTC) from {username='kudu', principal='kudu/127.24.227.254@KRBTEST.COM'} at 127.0.0.1:46986
I20260923 23:18:06.438280 10120 tablet_service.cc:1514] Processing DeleteTablet for tablet efa463299829459c8b8aba3ebde7a538 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 23:18:06 UTC) from {username='kudu', principal='kudu/127.24.227.254@KRBTEST.COM'} at 127.0.0.1:45760
I20260923 23:18:06.438427 10491 tablet_replica.cc:330] T efa463299829459c8b8aba3ebde7a538 P d20ee87894274b4aaa1b3f04353e9cf7: stopping tablet replica
I20260923 23:18:06.438439 10388 tablet_service.cc:1514] Processing DeleteTablet for tablet efa463299829459c8b8aba3ebde7a538 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 23:18:06 UTC) from {username='kudu', principal='kudu/127.24.227.254@KRBTEST.COM'} at 127.0.0.1:37972
I20260923 23:18:06.438545 10491 raft_consensus.cc:2229] T efa463299829459c8b8aba3ebde7a538 P d20ee87894274b4aaa1b3f04353e9cf7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 23:18:06.438637 10491 raft_consensus.cc:2258] T efa463299829459c8b8aba3ebde7a538 P d20ee87894274b4aaa1b3f04353e9cf7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 23:18:06.438632 10493 tablet_replica.cc:330] T efa463299829459c8b8aba3ebde7a538 P a4769a0ae42f4a6d90dd30080328cb6c: stopping tablet replica
I20260923 23:18:06.438791 10493 raft_consensus.cc:2229] T efa463299829459c8b8aba3ebde7a538 P a4769a0ae42f4a6d90dd30080328cb6c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 23:18:06.438896 10493 raft_consensus.cc:2258] T efa463299829459c8b8aba3ebde7a538 P a4769a0ae42f4a6d90dd30080328cb6c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 23:18:06.438910 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 10057
I20260923 23:18:06.439649 10491 ts_tablet_manager.cc:1905] T efa463299829459c8b8aba3ebde7a538 P d20ee87894274b4aaa1b3f04353e9cf7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 23:18:06.440001 10493 ts_tablet_manager.cc:1905] T efa463299829459c8b8aba3ebde7a538 P a4769a0ae42f4a6d90dd30080328cb6c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 23:18:06.441545 10491 ts_tablet_manager.cc:1918] T efa463299829459c8b8aba3ebde7a538 P d20ee87894274b4aaa1b3f04353e9cf7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 23:18:06.441629 10491 log.cc:1196] T efa463299829459c8b8aba3ebde7a538 P d20ee87894274b4aaa1b3f04353e9cf7: Deleting WAL directory at /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790205259943518-25487-0/minicluster-data/ts-1/wal/wals/efa463299829459c8b8aba3ebde7a538
I20260923 23:18:06.441854 10491 ts_tablet_manager.cc:1939] T efa463299829459c8b8aba3ebde7a538 P d20ee87894274b4aaa1b3f04353e9cf7: Deleting consensus metadata
I20260923 23:18:06.441855 10493 ts_tablet_manager.cc:1918] T efa463299829459c8b8aba3ebde7a538 P a4769a0ae42f4a6d90dd30080328cb6c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 23:18:06.441924 10493 log.cc:1196] T efa463299829459c8b8aba3ebde7a538 P a4769a0ae42f4a6d90dd30080328cb6c: Deleting WAL directory at /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790205259943518-25487-0/minicluster-data/ts-2/wal/wals/efa463299829459c8b8aba3ebde7a538
I20260923 23:18:06.442173 10493 ts_tablet_manager.cc:1939] T efa463299829459c8b8aba3ebde7a538 P a4769a0ae42f4a6d90dd30080328cb6c: Deleting consensus metadata
I20260923 23:18:06.442260 9979 catalog_manager.cc:4841] TS d20ee87894274b4aaa1b3f04353e9cf7 (127.24.227.194:43659): tablet efa463299829459c8b8aba3ebde7a538 (table test-table [id=9b9ce3a73d094450b4b97d4eaa287c0c]) successfully deleted
I20260923 23:18:06.442660 9982 catalog_manager.cc:4841] TS a4769a0ae42f4a6d90dd30080328cb6c (127.24.227.195:45575): tablet efa463299829459c8b8aba3ebde7a538 (table test-table [id=9b9ce3a73d094450b4b97d4eaa287c0c]) successfully deleted
W20260923 23:18:06.446544 9980 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.227.193:41327 (error 108)
I20260923 23:18:06.446861 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 10191
I20260923 23:18:06.447029 10463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:06.446903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.193:41327 (local address 127.0.0.1:45766)
0923 23:18:06.446961 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:06.446994 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.193:41327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260923 23:18:06.447508 9980 catalog_manager.cc:4584] TS 824c15d92f1446e0a6e9c1484170e3a1 (127.24.227.193:41327): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet efa463299829459c8b8aba3ebde7a538: Network error: Client connection negotiation failed: client connection to 127.24.227.193:41327: connect: Connection refused (error 111)
I20260923 23:18:06.452114 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 10325
I20260923 23:18:06.456658 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 9963
2026-09-23T23:18:06Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (1905 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-09-23T23:18:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T23:18:06Z Disabled control of system clock
I20260923 23:18:06.482769 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790205259943518-25487-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790205259943518-25487-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790205259943518-25487-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.254:37247
--webserver_interface=127.24.227.254
--webserver_port=0
--builtin_ntp_servers=127.24.227.212:42665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.227.254:37247
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260923 23:18:06.559406 10498 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:18:06.559600 10498 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:18:06.559621 10498 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:18:06.560784 10498 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 23:18:06.560823 10498 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:18:06.560837 10498 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:18:06.560848 10498 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 23:18:06.560899 10498 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 23:18:06.562206 10498 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:42665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790205259943518-25487-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790205259943518-25487-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.227.254:37247
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.254:37247
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.227.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10498
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790205259943518-25487-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:18:06.562449 10498 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:18:06.562733 10498 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:18:06.567214 10506 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:18:06.567238 10503 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:18:06.567226 10504 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:18:06.567485 10498 server_base.cc:1034] running on GCE node
I20260923 23:18:06.567706 10498 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:18:06.568005 10498 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:18:06.569164 10498 hybrid_clock.cc:648] HybridClock initialized: now 1790205486569147 us; error 33 us; skew 500 ppm
I20260923 23:18:06.570380 10498 webserver.cc:460] Webserver started at http://127.24.227.254:39197/ using document root <none> and password file <none>
I20260923 23:18:06.570614 10498 fs_manager.cc:359] Metadata directory not provided
I20260923 23:18:06.570660 10498 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:18:06.570807 10498 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:18:06.571789 10498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790205259943518-25487-0/minicluster-data/master-0/data/instance:
uuid: "2b39fbb7ccb04f518044ef8ea8fce44c"
format_stamp: "Formatted at 2026-09-23 23:18:06 on dist-test-slave-754n"
server_key: "87e4c65c1215787ab590fad8fd212576"
server_key_iv: "7d88c5fb1d1b88ffd8fe4ad735965408"
server_key_version: "encryptionkey@0"
I20260923 23:18:06.572204 10498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790205259943518-25487-0/minicluster-data/master-0/wal/instance:
uuid: "2b39fbb7ccb04f518044ef8ea8fce44c"
format_stamp: "Formatted at 2026-09-23 23:18:06 on dist-test-slave-754n"
server_key: "87e4c65c1215787ab590fad8fd212576"
server_key_iv: "7d88c5fb1d1b88ffd8fe4ad735965408"
server_key_version: "encryptionkey@0"
I20260923 23:18:06.573558 10498 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 23:18:06.574288 10512 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:18:06.574492 10498 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 23:18:06.574586 10498 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790205259943518-25487-0/minicluster-data/master-0/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790205259943518-25487-0/minicluster-data/master-0/wal
uuid: "2b39fbb7ccb04f518044ef8ea8fce44c"
format_stamp: "Formatted at 2026-09-23 23:18:06 on dist-test-slave-754n"
server_key: "87e4c65c1215787ab590fad8fd212576"
server_key_iv: "7d88c5fb1d1b88ffd8fe4ad735965408"
server_key_version: "encryptionkey@0"
I20260923 23:18:06.574671 10498 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790205259943518-25487-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790205259943518-25487-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790205259943518-25487-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:18:06.589607 10498 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:18:06.589884 10498 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:18:06.590047 10498 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:18:06.593832 10498 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.254:37247
I20260923 23:18:06.593859 10564 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.254:37247 every 8 connection(s)
I20260923 23:18:06.594146 10498 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
I20260923 23:18:06.594996 10565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:18:06.596577 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 10498
I20260923 23:18:06.596705 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790205259943518-25487-0/minicluster-data/master-0/wal/instance
I20260923 23:18:06.596903 25487 external_mini_cluster.cc:1428] Setting key adceec76383f52509fbad0f2d70b0f5c
I20260923 23:18:06.597194 10565 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b39fbb7ccb04f518044ef8ea8fce44c: Bootstrap starting.
I20260923 23:18:06.597951 10565 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2b39fbb7ccb04f518044ef8ea8fce44c: Neither blocks nor log segments found. Creating new log.
I20260923 23:18:06.598253 10565 log.cc:824] T 00000000000000000000000000000000 P 2b39fbb7ccb04f518044ef8ea8fce44c: Log is configured to *not* fsync() on all Append() calls
I20260923 23:18:06.598986 10565 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b39fbb7ccb04f518044ef8ea8fce44c: No bootstrap required, opened a new log
I20260923 23:18:06.601131 10565 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2b39fbb7ccb04f518044ef8ea8fce44c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b39fbb7ccb04f518044ef8ea8fce44c" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 37247 } }
I20260923 23:18:06.601289 10565 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2b39fbb7ccb04f518044ef8ea8fce44c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:18:06.601330 10565 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2b39fbb7ccb04f518044ef8ea8fce44c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b39fbb7ccb04f518044ef8ea8fce44c, State: Initialized, Role: FOLLOWER
I20260923 23:18:06.601508 10565 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2b39fbb7ccb04f518044ef8ea8fce44c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b39fbb7ccb04f518044ef8ea8fce44c" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 37247 } }
I20260923 23:18:06.601591 10565 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2b39fbb7ccb04f518044ef8ea8fce44c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 23:18:06.601634 10565 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2b39fbb7ccb04f518044ef8ea8fce44c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 23:18:06.601717 10565 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2b39fbb7ccb04f518044ef8ea8fce44c [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:18:06.602435 10565 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2b39fbb7ccb04f518044ef8ea8fce44c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b39fbb7ccb04f518044ef8ea8fce44c" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 37247 } }
I20260923 23:18:06.602545 10565 leader_election.cc:302] T 00000000000000000000000000000000 P 2b39fbb7ccb04f518044ef8ea8fce44c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2b39fbb7ccb04f518044ef8ea8fce44c; no voters:
I20260923 23:18:06.602692 10565 leader_election.cc:288] T 00000000000000000000000000000000 P 2b39fbb7ccb04f518044ef8ea8fce44c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 23:18:06.602761 10570 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2b39fbb7ccb04f518044ef8ea8fce44c [term 1 FOLLOWER]: Leader election won for term 1
I20260923 23:18:06.602782 10568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:06.597366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49470 (local address 127.24.227.254:37247)
0923 23:18:06.597574 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:06.597583 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:06.597619 (+ 36us) server_negotiation.cc:407] Connection header received
0923 23:18:06.598161 (+ 542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:06.598170 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:06.598248 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:06.598405 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:06.598598 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:06.599909 (+ 1311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:06.600640 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:06.601284 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:06.601417 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:06.602092 (+ 675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:18:06.602096 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:18:06.602097 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 23:18:06.602286 (+ 189us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:18:06.602341 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 23:18:06.602355 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:18:06.602428 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:06.602490 (+ 62us) server_negotiation.cc:299] Negotiation successful
0923 23:18:06.602564 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":95,"threads_started":1}
I20260923 23:18:06.602942 10570 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2b39fbb7ccb04f518044ef8ea8fce44c [term 1 LEADER]: Becoming Leader. State: Replica: 2b39fbb7ccb04f518044ef8ea8fce44c, State: Running, Role: LEADER
I20260923 23:18:06.602953 10565 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2b39fbb7ccb04f518044ef8ea8fce44c [sys.catalog]: configured and running, proceeding with master startup.
I20260923 23:18:06.604099 10570 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2b39fbb7ccb04f518044ef8ea8fce44c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b39fbb7ccb04f518044ef8ea8fce44c" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 37247 } }
I20260923 23:18:06.604705 10571 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b39fbb7ccb04f518044ef8ea8fce44c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2b39fbb7ccb04f518044ef8ea8fce44c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b39fbb7ccb04f518044ef8ea8fce44c" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 37247 } } }
I20260923 23:18:06.604686 10572 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b39fbb7ccb04f518044ef8ea8fce44c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2b39fbb7ccb04f518044ef8ea8fce44c. Latest consensus state: current_term: 1 leader_uuid: "2b39fbb7ccb04f518044ef8ea8fce44c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b39fbb7ccb04f518044ef8ea8fce44c" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 37247 } } }
I20260923 23:18:06.604828 10572 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b39fbb7ccb04f518044ef8ea8fce44c [sys.catalog]: This master's current role is: LEADER
I20260923 23:18:06.604827 10571 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b39fbb7ccb04f518044ef8ea8fce44c [sys.catalog]: This master's current role is: LEADER
W20260923 23:18:06.605345 10584 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2b39fbb7ccb04f518044ef8ea8fce44c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 23:18:06.605396 10584 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 23:18:06.605454 10586 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 23:18:06.605587 10586 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 23:18:06.606864 10586 catalog_manager.cc:1347] Generated new cluster ID: 07b4030f769441dba23e8e5e8a698645
I20260923 23:18:06.606904 10586 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 23:18:06.617712 10586 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 23:18:06.618219 10586 catalog_manager.cc:1504] Loading token signing keys...
I20260923 23:18:06.625962 10586 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2b39fbb7ccb04f518044ef8ea8fce44c: Generated new TSK 0
I20260923 23:18:06.626116 10586 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 23:18:06.628718 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790205259943518-25487-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790205259943518-25487-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790205259943518-25487-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790205259943518-25487-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.227.193:0
--local_ip_for_outbound_sockets=127.24.227.193
--webserver_interface=127.24.227.193
--webserver_port=0
--tserver_master_addrs=127.24.227.254:37247
--builtin_ntp_servers=127.24.227.212:42665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260923 23:18:06.710827 10589 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:18:06.711051 10589 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:18:06.711086 10589 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:18:06.712260 10589 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:18:06.712312 10589 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:18:06.712361 10589 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.227.193
I20260923 23:18:06.713675 10589 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:42665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790205259943518-25487-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790205259943518-25487-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.227.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790205259943518-25487-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.227.193
--webserver_port=0
--tserver_master_addrs=127.24.227.254:37247
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10589
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.227.193
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790205259943518-25487-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:18:06.713922 10589 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:18:06.714195 10589 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:18:06.719718 10597 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:18:06.719732 10595 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:18:06.719846 10589 server_base.cc:1034] running on GCE node
W20260923 23:18:06.720021 10594 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:18:06.720201 10589 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:18:06.720432 10589 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:18:06.721591 10589 hybrid_clock.cc:648] HybridClock initialized: now 1790205486721577 us; error 27 us; skew 500 ppm
I20260923 23:18:06.722795 10589 webserver.cc:460] Webserver started at http://127.24.227.193:36657/ using document root <none> and password file <none>
I20260923 23:18:06.723038 10589 fs_manager.cc:359] Metadata directory not provided
I20260923 23:18:06.723107 10589 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:18:06.723261 10589 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:18:06.724313 10589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790205259943518-25487-0/minicluster-data/ts-0/data/instance:
uuid: "b4dbb0679b1742808cff68ba15d0ee5b"
format_stamp: "Formatted at 2026-09-23 23:18:06 on dist-test-slave-754n"
server_key: "12175f3eb5840cc53a850b74834ec671"
server_key_iv: "c95f842d33e832aa969dab4abb1030ee"
server_key_version: "encryptionkey@0"
I20260923 23:18:06.724656 10589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790205259943518-25487-0/minicluster-data/ts-0/wal/instance:
uuid: "b4dbb0679b1742808cff68ba15d0ee5b"
format_stamp: "Formatted at 2026-09-23 23:18:06 on dist-test-slave-754n"
server_key: "12175f3eb5840cc53a850b74834ec671"
server_key_iv: "c95f842d33e832aa969dab4abb1030ee"
server_key_version: "encryptionkey@0"
I20260923 23:18:06.726014 10589 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260923 23:18:06.726792 10603 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:18:06.726970 10589 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 23:18:06.727063 10589 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790205259943518-25487-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790205259943518-25487-0/minicluster-data/ts-0/wal
uuid: "b4dbb0679b1742808cff68ba15d0ee5b"
format_stamp: "Formatted at 2026-09-23 23:18:06 on dist-test-slave-754n"
server_key: "12175f3eb5840cc53a850b74834ec671"
server_key_iv: "c95f842d33e832aa969dab4abb1030ee"
server_key_version: "encryptionkey@0"
I20260923 23:18:06.727144 10589 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790205259943518-25487-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790205259943518-25487-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790205259943518-25487-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:18:06.743079 10589 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:18:06.743352 10589 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:18:06.743508 10589 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:18:06.743670 10589 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 23:18:06.743989 10589 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 23:18:06.744043 10589 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 23:18:06.744077 10589 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 23:18:06.744125 10589 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 23:18:06.749562 10589 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.193:34445
I20260923 23:18:06.749631 10716 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.193:34445 every 8 connection(s)
I20260923 23:18:06.749933 10589 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790205259943518-25487-0/minicluster-data/ts-0/data/info.pb
I20260923 23:18:06.753685 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 10589
I20260923 23:18:06.753775 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790205259943518-25487-0/minicluster-data/ts-0/wal/instance
I20260923 23:18:06.753973 25487 external_mini_cluster.cc:1428] Setting key 383d75149fae26ef10af215ea964ec5b
I20260923 23:18:06.754915 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790205259943518-25487-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790205259943518-25487-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790205259943518-25487-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790205259943518-25487-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.227.194:0
--local_ip_for_outbound_sockets=127.24.227.194
--webserver_interface=127.24.227.194
--webserver_port=0
--tserver_master_addrs=127.24.227.254:37247
--builtin_ntp_servers=127.24.227.212:42665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260923 23:18:06.755393 10568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:06.750620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.193:34529 (local address 127.24.227.254:37247)
0923 23:18:06.750664 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:06.750666 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:06.750864 (+ 198us) server_negotiation.cc:407] Connection header received
0923 23:18:06.751353 (+ 489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:06.751354 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:06.751366 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:06.751428 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:06.752082 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:06.752585 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:06.753876 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:06.754124 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:06.754209 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:06.755131 (+ 922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:18:06.755132 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:18:06.755132 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 23:18:06.755137 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:18:06.755152 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 23:18:06.755156 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:18:06.755209 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:06.755282 (+ 73us) server_negotiation.cc:299] Negotiation successful
0923 23:18:06.755309 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260923 23:18:06.755625 10719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:06.750576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:37247 (local address 127.24.227.193:34529)
0923 23:18:06.750735 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:06.750774 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0923 23:18:06.751248 (+ 474us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:18:06.751492 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:18:06.751496 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:18:06.751506 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:18:06.751990 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:06.751994 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:06.752680 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:06.752681 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:06.753454 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:06.753456 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:06.754229 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:06.754230 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:06.754394 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:06.754977 (+ 583us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 23:18:06.754978 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:18:06.755013 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 23:18:06.755034 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:18:06.755210 (+ 176us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:18:06.755216 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:18:06.755218 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 23:18:06.755309 (+ 91us) client_negotiation.cc:240] Negotiation successful
0923 23:18:06.755409 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":67,"threads_started":1}
I20260923 23:18:06.756117 10717 heartbeater.cc:347] Connected to a master server at 127.24.227.254:37247
I20260923 23:18:06.756227 10717 heartbeater.cc:464] Registering TS with master...
I20260923 23:18:06.756448 10717 heartbeater.cc:511] Master 127.24.227.254:37247 requested a full tablet report, sending...
I20260923 23:18:06.756966 10529 ts_manager.cc:194] Registered new tserver with Master: b4dbb0679b1742808cff68ba15d0ee5b (127.24.227.193:34445)
I20260923 23:18:06.757905 10529 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.227.193:34529
W20260923 23:18:06.834856 10720 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:18:06.835034 10720 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:18:06.835054 10720 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:18:06.836107 10720 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:18:06.836134 10720 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:18:06.836156 10720 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.227.194
I20260923 23:18:06.837306 10720 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:42665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790205259943518-25487-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790205259943518-25487-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.227.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790205259943518-25487-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.227.194
--webserver_port=0
--tserver_master_addrs=127.24.227.254:37247
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10720
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.227.194
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790205259943518-25487-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:18:06.837468 10720 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:18:06.837704 10720 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:18:06.842507 10728 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:18:06.842566 10725 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:18:06.842536 10726 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:18:06.842721 10720 server_base.cc:1034] running on GCE node
I20260923 23:18:06.842902 10720 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:18:06.843102 10720 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:18:06.844288 10720 hybrid_clock.cc:648] HybridClock initialized: now 1790205486844271 us; error 33 us; skew 500 ppm
I20260923 23:18:06.845403 10720 webserver.cc:460] Webserver started at http://127.24.227.194:39477/ using document root <none> and password file <none>
I20260923 23:18:06.845618 10720 fs_manager.cc:359] Metadata directory not provided
I20260923 23:18:06.845695 10720 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:18:06.845860 10720 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:18:06.846824 10720 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790205259943518-25487-0/minicluster-data/ts-1/data/instance:
uuid: "70b6dda309944660acf247e192b91913"
format_stamp: "Formatted at 2026-09-23 23:18:06 on dist-test-slave-754n"
server_key: "974b5d186fa9a5be997950c6a0d6dcaa"
server_key_iv: "b40fffda8cb5ac4412ac0ee37a9ce710"
server_key_version: "encryptionkey@0"
I20260923 23:18:06.847168 10720 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790205259943518-25487-0/minicluster-data/ts-1/wal/instance:
uuid: "70b6dda309944660acf247e192b91913"
format_stamp: "Formatted at 2026-09-23 23:18:06 on dist-test-slave-754n"
server_key: "974b5d186fa9a5be997950c6a0d6dcaa"
server_key_iv: "b40fffda8cb5ac4412ac0ee37a9ce710"
server_key_version: "encryptionkey@0"
I20260923 23:18:06.848548 10720 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 23:18:06.849270 10734 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:18:06.849462 10720 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 23:18:06.849563 10720 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790205259943518-25487-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790205259943518-25487-0/minicluster-data/ts-1/wal
uuid: "70b6dda309944660acf247e192b91913"
format_stamp: "Formatted at 2026-09-23 23:18:06 on dist-test-slave-754n"
server_key: "974b5d186fa9a5be997950c6a0d6dcaa"
server_key_iv: "b40fffda8cb5ac4412ac0ee37a9ce710"
server_key_version: "encryptionkey@0"
I20260923 23:18:06.849656 10720 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790205259943518-25487-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790205259943518-25487-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790205259943518-25487-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:18:06.877620 10720 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:18:06.877923 10720 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:18:06.878067 10720 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:18:06.878298 10720 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 23:18:06.878612 10720 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 23:18:06.878665 10720 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 23:18:06.878707 10720 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 23:18:06.878731 10720 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 23:18:06.884465 10720 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.194:33277
I20260923 23:18:06.884506 10847 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.194:33277 every 8 connection(s)
I20260923 23:18:06.884804 10720 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790205259943518-25487-0/minicluster-data/ts-1/data/info.pb
I20260923 23:18:06.889493 10568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:06.885418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.194:33585 (local address 127.24.227.254:37247)
0923 23:18:06.885459 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:06.885461 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:06.885691 (+ 230us) server_negotiation.cc:407] Connection header received
0923 23:18:06.886173 (+ 482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:06.886174 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:06.886183 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:06.886241 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:06.886873 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:06.887357 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:06.888353 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:06.888533 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:06.888581 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:06.889278 (+ 697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:18:06.889279 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:18:06.889280 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 23:18:06.889283 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:18:06.889296 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 23:18:06.889300 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:18:06.889349 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:06.889419 (+ 70us) server_negotiation.cc:299] Negotiation successful
0923 23:18:06.889439 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260923 23:18:06.889669 10850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:06.885392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:37247 (local address 127.24.227.194:33585)
0923 23:18:06.885579 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:06.885619 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0923 23:18:06.886092 (+ 473us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:18:06.886312 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:18:06.886316 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:18:06.886325 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:18:06.886789 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:06.886793 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:06.887432 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:06.887433 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:06.888270 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:06.888273 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:06.888597 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:06.888598 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:06.888701 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:06.889146 (+ 445us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 23:18:06.889146 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:18:06.889172 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 23:18:06.889192 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:18:06.889353 (+ 161us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:18:06.889357 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:18:06.889359 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 23:18:06.889408 (+ 49us) client_negotiation.cc:240] Negotiation successful
0923 23:18:06.889480 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":95,"threads_started":1}
I20260923 23:18:06.890182 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 10720
I20260923 23:18:06.890218 10848 heartbeater.cc:347] Connected to a master server at 127.24.227.254:37247
I20260923 23:18:06.890285 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790205259943518-25487-0/minicluster-data/ts-1/wal/instance
I20260923 23:18:06.890327 10848 heartbeater.cc:464] Registering TS with master...
I20260923 23:18:06.890494 10848 heartbeater.cc:511] Master 127.24.227.254:37247 requested a full tablet report, sending...
I20260923 23:18:06.890499 25487 external_mini_cluster.cc:1428] Setting key bd61773245838f94b3537aec8afcf680
I20260923 23:18:06.890928 10528 ts_manager.cc:194] Registered new tserver with Master: 70b6dda309944660acf247e192b91913 (127.24.227.194:33277)
I20260923 23:18:06.891350 10528 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.227.194:33585
I20260923 23:18:06.891405 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790205259943518-25487-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790205259943518-25487-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790205259943518-25487-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790205259943518-25487-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.227.195:0
--local_ip_for_outbound_sockets=127.24.227.195
--webserver_interface=127.24.227.195
--webserver_port=0
--tserver_master_addrs=127.24.227.254:37247
--builtin_ntp_servers=127.24.227.212:42665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260923 23:18:06.970036 10851 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:18:06.970238 10851 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:18:06.970285 10851 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:18:06.971486 10851 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:18:06.971546 10851 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:18:06.971590 10851 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.227.195
I20260923 23:18:06.972982 10851 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:42665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790205259943518-25487-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790205259943518-25487-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.227.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790205259943518-25487-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.227.195
--webserver_port=0
--tserver_master_addrs=127.24.227.254:37247
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10851
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.227.195
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790205259943518-25487-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:18:06.973224 10851 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:18:06.973496 10851 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:18:06.978238 10859 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:18:06.978250 10857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:18:06.978252 10856 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:18:06.978410 10851 server_base.cc:1034] running on GCE node
I20260923 23:18:06.978662 10851 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:18:06.978888 10851 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:18:06.980037 10851 hybrid_clock.cc:648] HybridClock initialized: now 1790205486980020 us; error 30 us; skew 500 ppm
I20260923 23:18:06.981168 10851 webserver.cc:460] Webserver started at http://127.24.227.195:46245/ using document root <none> and password file <none>
I20260923 23:18:06.981413 10851 fs_manager.cc:359] Metadata directory not provided
I20260923 23:18:06.981488 10851 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:18:06.981632 10851 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:18:06.982614 10851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790205259943518-25487-0/minicluster-data/ts-2/data/instance:
uuid: "8741179c89ea46c1b31edb6d1e63be3c"
format_stamp: "Formatted at 2026-09-23 23:18:06 on dist-test-slave-754n"
server_key: "f8ec766fd77e4a93a663f2f71839c26c"
server_key_iv: "e3282e4f73d0e018072e523d39ffa0f3"
server_key_version: "encryptionkey@0"
I20260923 23:18:06.982968 10851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790205259943518-25487-0/minicluster-data/ts-2/wal/instance:
uuid: "8741179c89ea46c1b31edb6d1e63be3c"
format_stamp: "Formatted at 2026-09-23 23:18:06 on dist-test-slave-754n"
server_key: "f8ec766fd77e4a93a663f2f71839c26c"
server_key_iv: "e3282e4f73d0e018072e523d39ffa0f3"
server_key_version: "encryptionkey@0"
I20260923 23:18:06.984218 10851 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 23:18:06.984884 10865 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:18:06.985059 10851 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 23:18:06.985150 10851 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790205259943518-25487-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790205259943518-25487-0/minicluster-data/ts-2/wal
uuid: "8741179c89ea46c1b31edb6d1e63be3c"
format_stamp: "Formatted at 2026-09-23 23:18:06 on dist-test-slave-754n"
server_key: "f8ec766fd77e4a93a663f2f71839c26c"
server_key_iv: "e3282e4f73d0e018072e523d39ffa0f3"
server_key_version: "encryptionkey@0"
I20260923 23:18:06.985229 10851 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790205259943518-25487-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790205259943518-25487-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790205259943518-25487-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:18:06.996691 10851 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:18:06.996956 10851 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:18:06.997102 10851 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:18:06.997344 10851 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 23:18:06.997635 10851 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 23:18:06.997685 10851 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 23:18:06.997725 10851 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 23:18:06.997753 10851 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 23:18:07.003221 10851 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.195:46693
I20260923 23:18:07.003293 10978 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.195:46693 every 8 connection(s)
I20260923 23:18:07.003587 10851 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790205259943518-25487-0/minicluster-data/ts-2/data/info.pb
I20260923 23:18:07.005818 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 10851
I20260923 23:18:07.005941 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790205259943518-25487-0/minicluster-data/ts-2/wal/instance
I20260923 23:18:07.006112 25487 external_mini_cluster.cc:1428] Setting key d2c65c45fd5460b98c49d8dd3213e846
I20260923 23:18:07.008841 10568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:07.004375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.195:33085 (local address 127.24.227.254:37247)
0923 23:18:07.004436 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:07.004438 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:07.004622 (+ 184us) server_negotiation.cc:407] Connection header received
0923 23:18:07.005030 (+ 408us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:07.005030 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:07.005039 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:07.005091 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:07.005669 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:07.006255 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:07.007405 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:07.007680 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:07.007756 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:07.008612 (+ 856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:18:07.008613 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:18:07.008613 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 23:18:07.008617 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:18:07.008630 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 23:18:07.008634 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:18:07.008677 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:07.008743 (+ 66us) server_negotiation.cc:299] Negotiation successful
0923 23:18:07.008767 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260923 23:18:07.009047 10981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:07.004334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:37247 (local address 127.24.227.195:33085)
0923 23:18:07.004503 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:07.004538 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0923 23:18:07.004956 (+ 418us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:18:07.005135 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:18:07.005138 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:18:07.005145 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:18:07.005579 (+ 434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:07.005583 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:07.006391 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:07.006392 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:07.007298 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:07.007301 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:07.007760 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:07.007761 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:07.007948 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:07.008472 (+ 524us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 23:18:07.008472 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:18:07.008498 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 23:18:07.008517 (+ 19us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:18:07.008676 (+ 159us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:18:07.008681 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:18:07.008684 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 23:18:07.008745 (+ 61us) client_negotiation.cc:240] Negotiation successful
0923 23:18:07.008842 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":104,"threads_started":1}
I20260923 23:18:07.009485 10979 heartbeater.cc:347] Connected to a master server at 127.24.227.254:37247
I20260923 23:18:07.009552 10979 heartbeater.cc:464] Registering TS with master...
I20260923 23:18:07.009760 10979 heartbeater.cc:511] Master 127.24.227.254:37247 requested a full tablet report, sending...
I20260923 23:18:07.010130 10528 ts_manager.cc:194] Registered new tserver with Master: 8741179c89ea46c1b31edb6d1e63be3c (127.24.227.195:46693)
I20260923 23:18:07.010525 10528 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.227.195:33085
I20260923 23:18:07.017885 25487 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 23:18:07.022019 10568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:07.019208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49480 (local address 127.24.227.254:37247)
0923 23:18:07.019275 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:07.019277 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:07.019376 (+ 99us) server_negotiation.cc:407] Connection header received
0923 23:18:07.019449 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:07.019450 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:07.019459 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:07.019525 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:07.019677 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:07.020180 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:07.020905 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:07.021111 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:07.021185 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:07.021806 (+ 621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:18:07.021807 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:18:07.021808 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 23:18:07.021812 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:18:07.021824 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 23:18:07.021828 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:18:07.021876 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:07.021925 (+ 49us) server_negotiation.cc:299] Negotiation successful
0923 23:18:07.021944 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260923 23:18:07.023456 10528 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49480:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 23:18:07.024986 10528 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 23:18:07.030967 10987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:07.026986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.194:33277 (local address 127.0.0.1:55392)
0923 23:18:07.027367 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:07.027377 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 23:18:07.027443 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:18:07.027720 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:18:07.027721 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:18:07.027723 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:18:07.027759 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:07.027760 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:07.028946 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:07.028947 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:07.029941 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:07.029944 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:07.030678 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:07.030679 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:07.030751 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:07.030764 (+ 13us) client_negotiation.cc:769] Sending connection context
0923 23:18:07.030846 (+ 82us) client_negotiation.cc:240] Negotiation successful
0923 23:18:07.030896 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":51,"threads_started":1}
I20260923 23:18:07.030983 10992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:07.027163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55392 (local address 127.24.227.194:33277)
0923 23:18:07.027316 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:07.027320 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:07.027520 (+ 200us) server_negotiation.cc:407] Connection header received
0923 23:18:07.027566 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:07.027568 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:07.027584 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:07.027655 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:18:07.027830 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:07.028883 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:07.030047 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:07.030574 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:07.030677 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:07.030764 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:07.030861 (+ 97us) server_negotiation.cc:299] Negotiation successful
0923 23:18:07.030901 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":93,"threads_started":1}
I20260923 23:18:07.031173 10988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:07.026811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.195:46693 (local address 127.0.0.1:57944)
0923 23:18:07.026992 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:07.027032 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0923 23:18:07.027093 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:18:07.027293 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:18:07.027296 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:18:07.027298 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:18:07.027366 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:07.027368 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:07.028545 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:07.028546 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:07.029834 (+ 1288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:07.029856 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:07.030916 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:07.030917 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:07.031017 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:07.031021 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 23:18:07.031068 (+ 47us) client_negotiation.cc:240] Negotiation successful
0923 23:18:07.031104 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"mutex_wait_us":3,"thread_start_us":47,"threads_started":1}
I20260923 23:18:07.031198 10990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:07.026936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38720 (local address 127.24.227.193:34445)
0923 23:18:07.027109 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:07.027114 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:07.027432 (+ 318us) server_negotiation.cc:407] Connection header received
0923 23:18:07.027507 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:07.027509 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:07.027524 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:07.027592 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:18:07.027698 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:07.028808 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:07.030064 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:07.030729 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:07.030802 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:07.030885 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:07.031077 (+ 192us) server_negotiation.cc:299] Negotiation successful
0923 23:18:07.031115 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":95,"threads_started":1}
I20260923 23:18:07.031384 10991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:07.026847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.193:34445 (local address 127.0.0.1:38720)
0923 23:18:07.027353 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:07.027362 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 23:18:07.027440 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:18:07.027599 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:18:07.027599 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:18:07.027601 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:18:07.027647 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:07.027649 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:07.028909 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:07.028910 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:07.030004 (+ 1094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:07.030006 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:07.030917 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:07.030918 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:07.030982 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:07.030986 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 23:18:07.031290 (+ 304us) client_negotiation.cc:240] Negotiation successful
0923 23:18:07.031326 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":81,"threads_started":1}
I20260923 23:18:07.031383 10989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:07.026863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57944 (local address 127.24.227.195:46693)
0923 23:18:07.027141 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:07.027146 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:07.027156 (+ 10us) server_negotiation.cc:407] Connection header received
0923 23:18:07.027189 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:07.027191 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:07.027208 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:07.027290 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:18:07.027451 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:07.028455 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:07.029960 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:07.030729 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:07.030801 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:07.030877 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:07.031263 (+ 386us) server_negotiation.cc:299] Negotiation successful
0923 23:18:07.031301 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":246,"thread_start_us":65,"threads_started":1}
I20260923 23:18:07.031792 10782 tablet_service.cc:1467] Processing CreateTablet for tablet 356cc4c8edaa40a685e7e2fd05c3b7c4 (DEFAULT_TABLE table=test-table [id=6e39d830e6c54fbbaf6e957d82e43270]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 23:18:07.031973 10913 tablet_service.cc:1467] Processing CreateTablet for tablet 356cc4c8edaa40a685e7e2fd05c3b7c4 (DEFAULT_TABLE table=test-table [id=6e39d830e6c54fbbaf6e957d82e43270]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 23:18:07.032182 10782 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 356cc4c8edaa40a685e7e2fd05c3b7c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:18:07.032232 10913 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 356cc4c8edaa40a685e7e2fd05c3b7c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:18:07.032400 10651 tablet_service.cc:1467] Processing CreateTablet for tablet 356cc4c8edaa40a685e7e2fd05c3b7c4 (DEFAULT_TABLE table=test-table [id=6e39d830e6c54fbbaf6e957d82e43270]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 23:18:07.032653 10651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 356cc4c8edaa40a685e7e2fd05c3b7c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:18:07.034816 10993 tablet_bootstrap.cc:492] T 356cc4c8edaa40a685e7e2fd05c3b7c4 P b4dbb0679b1742808cff68ba15d0ee5b: Bootstrap starting.
I20260923 23:18:07.035434 10993 tablet_bootstrap.cc:654] T 356cc4c8edaa40a685e7e2fd05c3b7c4 P b4dbb0679b1742808cff68ba15d0ee5b: Neither blocks nor log segments found. Creating new log.
I20260923 23:18:07.035449 10995 tablet_bootstrap.cc:492] T 356cc4c8edaa40a685e7e2fd05c3b7c4 P 70b6dda309944660acf247e192b91913: Bootstrap starting.
I20260923 23:18:07.035743 10993 log.cc:824] T 356cc4c8edaa40a685e7e2fd05c3b7c4 P b4dbb0679b1742808cff68ba15d0ee5b: Log is configured to *not* fsync() on all Append() calls
I20260923 23:18:07.036000 10994 tablet_bootstrap.cc:492] T 356cc4c8edaa40a685e7e2fd05c3b7c4 P 8741179c89ea46c1b31edb6d1e63be3c: Bootstrap starting.
I20260923 23:18:07.036074 10995 tablet_bootstrap.cc:654] T 356cc4c8edaa40a685e7e2fd05c3b7c4 P 70b6dda309944660acf247e192b91913: Neither blocks nor log segments found. Creating new log.
I20260923 23:18:07.036410 10995 log.cc:824] T 356cc4c8edaa40a685e7e2fd05c3b7c4 P 70b6dda309944660acf247e192b91913: Log is configured to *not* fsync() on all Append() calls
I20260923 23:18:07.036518 10993 tablet_bootstrap.cc:492] T 356cc4c8edaa40a685e7e2fd05c3b7c4 P b4dbb0679b1742808cff68ba15d0ee5b: No bootstrap required, opened a new log
I20260923 23:18:07.036589 10993 ts_tablet_manager.cc:1397] T 356cc4c8edaa40a685e7e2fd05c3b7c4 P b4dbb0679b1742808cff68ba15d0ee5b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 23:18:07.036612 10994 tablet_bootstrap.cc:654] T 356cc4c8edaa40a685e7e2fd05c3b7c4 P 8741179c89ea46c1b31edb6d1e63be3c: Neither blocks nor log segments found. Creating new log.
I20260923 23:18:07.036928 10994 log.cc:824] T 356cc4c8edaa40a685e7e2fd05c3b7c4 P 8741179c89ea46c1b31edb6d1e63be3c: Log is configured to *not* fsync() on all Append() calls
I20260923 23:18:07.037073 10995 tablet_bootstrap.cc:492] T 356cc4c8edaa40a685e7e2fd05c3b7c4 P 70b6dda309944660acf247e192b91913: No bootstrap required, opened a new log
I20260923 23:18:07.037158 10995 ts_tablet_manager.cc:1397] T 356cc4c8edaa40a685e7e2fd05c3b7c4 P 70b6dda309944660acf247e192b91913: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 23:18:07.037760 10994 tablet_bootstrap.cc:492] T 356cc4c8edaa40a685e7e2fd05c3b7c4 P 8741179c89ea46c1b31edb6d1e63be3c: No bootstrap required, opened a new log
I20260923 23:18:07.037811 10994 ts_tablet_manager.cc:1397] T 356cc4c8edaa40a685e7e2fd05c3b7c4 P 8741179c89ea46c1b31edb6d1e63be3c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 23:18:07.039191 10995 raft_consensus.cc:348] T 356cc4c8edaa40a685e7e2fd05c3b7c4 P 70b6dda309944660acf247e192b91913 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8741179c89ea46c1b31edb6d1e63be3c" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 46693 } } peers { permanent_uuid: "70b6dda309944660acf247e192b91913" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 33277 } } peers { permanent_uuid: "b4dbb0679b1742808cff68ba15d0ee5b" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 34445 } }
I20260923 23:18:07.039299 10995 raft_consensus.cc:374] T 356cc4c8edaa40a685e7e2fd05c3b7c4 P 70b6dda309944660acf247e192b91913 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:18:07.039322 10995 raft_consensus.cc:729] T 356cc4c8edaa40a685e7e2fd05c3b7c4 P 70b6dda309944660acf247e192b91913 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70b6dda309944660acf247e192b91913, State: Initialized, Role: FOLLOWER
I20260923 23:18:07.039256 10993 raft_consensus.cc:348] T 356cc4c8edaa40a685e7e2fd05c3b7c4 P b4dbb0679b1742808cff68ba15d0ee5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8741179c89ea46c1b31edb6d1e63be3c" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 46693 } } peers { permanent_uuid: "70b6dda309944660acf247e192b91913" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 33277 } } peers { permanent_uuid: "b4dbb0679b1742808cff68ba15d0ee5b" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 34445 } }
I20260923 23:18:07.039409 10993 raft_consensus.cc:374] T 356cc4c8edaa40a685e7e2fd05c3b7c4 P b4dbb0679b1742808cff68ba15d0ee5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:18:07.039444 10993 raft_consensus.cc:729] T 356cc4c8edaa40a685e7e2fd05c3b7c4 P b4dbb0679b1742808cff68ba15d0ee5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4dbb0679b1742808cff68ba15d0ee5b, State: Initialized, Role: FOLLOWER
I20260923 23:18:07.039433 10995 consensus_queue.cc:260] T 356cc4c8edaa40a685e7e2fd05c3b7c4 P 70b6dda309944660acf247e192b91913 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8741179c89ea46c1b31edb6d1e63be3c" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 46693 } } peers { permanent_uuid: "70b6dda309944660acf247e192b91913" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 33277 } } peers { permanent_uuid: "b4dbb0679b1742808cff68ba15d0ee5b" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 34445 } }
I20260923 23:18:07.039557 10993 consensus_queue.cc:260] T 356cc4c8edaa40a685e7e2fd05c3b7c4 P b4dbb0679b1742808cff68ba15d0ee5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8741179c89ea46c1b31edb6d1e63be3c" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 46693 } } peers { permanent_uuid: "70b6dda309944660acf247e192b91913" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 33277 } } peers { permanent_uuid: "b4dbb0679b1742808cff68ba15d0ee5b" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 34445 } }
I20260923 23:18:07.039657 10995 ts_tablet_manager.cc:1428] T 356cc4c8edaa40a685e7e2fd05c3b7c4 P 70b6dda309944660acf247e192b91913: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260923 23:18:07.039772 10848 heartbeater.cc:503] Master 127.24.227.254:37247 was elected leader, sending a full tablet report...
I20260923 23:18:07.039801 10993 ts_tablet_manager.cc:1428] T 356cc4c8edaa40a685e7e2fd05c3b7c4 P b4dbb0679b1742808cff68ba15d0ee5b: Time spent starting tablet: real 0.003s user 0.000s sys 0.004s
I20260923 23:18:07.040042 10717 heartbeater.cc:503] Master 127.24.227.254:37247 was elected leader, sending a full tablet report...
I20260923 23:18:07.040320 10994 raft_consensus.cc:348] T 356cc4c8edaa40a685e7e2fd05c3b7c4 P 8741179c89ea46c1b31edb6d1e63be3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8741179c89ea46c1b31edb6d1e63be3c" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 46693 } } peers { permanent_uuid: "70b6dda309944660acf247e192b91913" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 33277 } } peers { permanent_uuid: "b4dbb0679b1742808cff68ba15d0ee5b" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 34445 } }
I20260923 23:18:07.040460 10994 raft_consensus.cc:374] T 356cc4c8edaa40a685e7e2fd05c3b7c4 P 8741179c89ea46c1b31edb6d1e63be3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:18:07.040494 10994 raft_consensus.cc:729] T 356cc4c8edaa40a685e7e2fd05c3b7c4 P 8741179c89ea46c1b31edb6d1e63be3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8741179c89ea46c1b31edb6d1e63be3c, State: Initialized, Role: FOLLOWER
I20260923 23:18:07.040623 10994 consensus_queue.cc:260] T 356cc4c8edaa40a685e7e2fd05c3b7c4 P 8741179c89ea46c1b31edb6d1e63be3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8741179c89ea46c1b31edb6d1e63be3c" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 46693 } } peers { permanent_uuid: "70b6dda309944660acf247e192b91913" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 33277 } } peers { permanent_uuid: "b4dbb0679b1742808cff68ba15d0ee5b" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 34445 } }
I20260923 23:18:07.040869 10994 ts_tablet_manager.cc:1428] T 356cc4c8edaa40a685e7e2fd05c3b7c4 P 8741179c89ea46c1b31edb6d1e63be3c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 23:18:07.041009 10979 heartbeater.cc:503] Master 127.24.227.254:37247 was elected leader, sending a full tablet report...
I20260923 23:18:07.121874 11000 raft_consensus.cc:482] T 356cc4c8edaa40a685e7e2fd05c3b7c4 P b4dbb0679b1742808cff68ba15d0ee5b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 23:18:07.122004 11000 raft_consensus.cc:504] T 356cc4c8edaa40a685e7e2fd05c3b7c4 P b4dbb0679b1742808cff68ba15d0ee5b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8741179c89ea46c1b31edb6d1e63be3c" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 46693 } } peers { permanent_uuid: "70b6dda309944660acf247e192b91913" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 33277 } } peers { permanent_uuid: "b4dbb0679b1742808cff68ba15d0ee5b" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 34445 } }
I20260923 23:18:07.122370 11000 leader_election.cc:288] T 356cc4c8edaa40a685e7e2fd05c3b7c4 P b4dbb0679b1742808cff68ba15d0ee5b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8741179c89ea46c1b31edb6d1e63be3c (127.24.227.195:46693), 70b6dda309944660acf247e192b91913 (127.24.227.194:33277)
I20260923 23:18:07.126168 10992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:07.122715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.193:35995 (local address 127.24.227.194:33277)
0923 23:18:07.122757 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:07.122759 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:07.122826 (+ 67us) server_negotiation.cc:407] Connection header received
0923 23:18:07.122919 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:07.122920 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:07.122926 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:07.122965 (+ 39us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:18:07.123093 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:07.123772 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:07.125272 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:07.125818 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:07.125894 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:07.125901 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:07.126061 (+ 160us) server_negotiation.cc:299] Negotiation successful
0923 23:18:07.126099 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260923 23:18:07.126168 10989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:07.122712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.193:34447 (local address 127.24.227.195:46693)
0923 23:18:07.122756 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:07.122758 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:07.122766 (+ 8us) server_negotiation.cc:407] Connection header received
0923 23:18:07.122791 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:07.122791 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:07.122799 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:07.122857 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:18:07.123021 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:07.123752 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:07.125271 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:07.125827 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:07.125894 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:07.125901 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:07.126061 (+ 160us) server_negotiation.cc:299] Negotiation successful
0923 23:18:07.126099 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260923 23:18:07.126187 10719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:07.122524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.195:46693 (local address 127.24.227.193:34447)
0923 23:18:07.122571 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:07.122582 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 23:18:07.122622 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:18:07.122886 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:18:07.122887 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:18:07.122889 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:18:07.122955 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:07.122957 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:07.123854 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:07.123855 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:07.125183 (+ 1328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:07.125186 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:07.125903 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:07.125904 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:07.125977 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:07.125981 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 23:18:07.126042 (+ 61us) client_negotiation.cc:240] Negotiation successful
0923 23:18:07.126096 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27,"mutex_wait_us":87}
I20260923 23:18:07.126194 11002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:07.122637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.194:33277 (local address 127.24.227.193:35995)
0923 23:18:07.122781 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:07.122789 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 23:18:07.122836 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:18:07.122969 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:18:07.122970 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:18:07.122971 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:18:07.123028 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:07.123029 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:07.123839 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:07.123840 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:07.125188 (+ 1348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:07.125190 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:07.125905 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:07.125905 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:07.125981 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:07.125990 (+ 9us) client_negotiation.cc:769] Sending connection context
0923 23:18:07.126042 (+ 52us) client_negotiation.cc:240] Negotiation successful
0923 23:18:07.126094 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"mutex_wait_us":78,"thread_start_us":82,"threads_started":1}
I20260923 23:18:07.126534 10933 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "356cc4c8edaa40a685e7e2fd05c3b7c4" candidate_uuid: "b4dbb0679b1742808cff68ba15d0ee5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8741179c89ea46c1b31edb6d1e63be3c" is_pre_election: true
I20260923 23:18:07.126550 10802 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "356cc4c8edaa40a685e7e2fd05c3b7c4" candidate_uuid: "b4dbb0679b1742808cff68ba15d0ee5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70b6dda309944660acf247e192b91913" is_pre_election: true
I20260923 23:18:07.126673 10802 raft_consensus.cc:2454] T 356cc4c8edaa40a685e7e2fd05c3b7c4 P 70b6dda309944660acf247e192b91913 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4dbb0679b1742808cff68ba15d0ee5b in term 0.
I20260923 23:18:07.126673 10933 raft_consensus.cc:2454] T 356cc4c8edaa40a685e7e2fd05c3b7c4 P 8741179c89ea46c1b31edb6d1e63be3c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4dbb0679b1742808cff68ba15d0ee5b in term 0.
I20260923 23:18:07.126891 10606 leader_election.cc:302] T 356cc4c8edaa40a685e7e2fd05c3b7c4 P b4dbb0679b1742808cff68ba15d0ee5b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8741179c89ea46c1b31edb6d1e63be3c, b4dbb0679b1742808cff68ba15d0ee5b; no voters:
I20260923 23:18:07.127008 11000 raft_consensus.cc:2781] T 356cc4c8edaa40a685e7e2fd05c3b7c4 P b4dbb0679b1742808cff68ba15d0ee5b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 23:18:07.127061 11000 raft_consensus.cc:482] T 356cc4c8edaa40a685e7e2fd05c3b7c4 P b4dbb0679b1742808cff68ba15d0ee5b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 23:18:07.127089 11000 raft_consensus.cc:3037] T 356cc4c8edaa40a685e7e2fd05c3b7c4 P b4dbb0679b1742808cff68ba15d0ee5b [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:18:07.127731 11000 raft_consensus.cc:504] T 356cc4c8edaa40a685e7e2fd05c3b7c4 P b4dbb0679b1742808cff68ba15d0ee5b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8741179c89ea46c1b31edb6d1e63be3c" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 46693 } } peers { permanent_uuid: "70b6dda309944660acf247e192b91913" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 33277 } } peers { permanent_uuid: "b4dbb0679b1742808cff68ba15d0ee5b" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 34445 } }
I20260923 23:18:07.127857 11000 leader_election.cc:288] T 356cc4c8edaa40a685e7e2fd05c3b7c4 P b4dbb0679b1742808cff68ba15d0ee5b [CANDIDATE]: Term 1 election: Requested vote from peers 8741179c89ea46c1b31edb6d1e63be3c (127.24.227.195:46693), 70b6dda309944660acf247e192b91913 (127.24.227.194:33277)
I20260923 23:18:07.128051 10933 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "356cc4c8edaa40a685e7e2fd05c3b7c4" candidate_uuid: "b4dbb0679b1742808cff68ba15d0ee5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8741179c89ea46c1b31edb6d1e63be3c"
I20260923 23:18:07.128057 10802 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "356cc4c8edaa40a685e7e2fd05c3b7c4" candidate_uuid: "b4dbb0679b1742808cff68ba15d0ee5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70b6dda309944660acf247e192b91913"
I20260923 23:18:07.128120 10933 raft_consensus.cc:3037] T 356cc4c8edaa40a685e7e2fd05c3b7c4 P 8741179c89ea46c1b31edb6d1e63be3c [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:18:07.128146 10802 raft_consensus.cc:3037] T 356cc4c8edaa40a685e7e2fd05c3b7c4 P 70b6dda309944660acf247e192b91913 [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:18:07.128803 10933 raft_consensus.cc:2454] T 356cc4c8edaa40a685e7e2fd05c3b7c4 P 8741179c89ea46c1b31edb6d1e63be3c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4dbb0679b1742808cff68ba15d0ee5b in term 1.
I20260923 23:18:07.128964 10606 leader_election.cc:302] T 356cc4c8edaa40a685e7e2fd05c3b7c4 P b4dbb0679b1742808cff68ba15d0ee5b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8741179c89ea46c1b31edb6d1e63be3c, b4dbb0679b1742808cff68ba15d0ee5b; no voters:
I20260923 23:18:07.128984 10802 raft_consensus.cc:2454] T 356cc4c8edaa40a685e7e2fd05c3b7c4 P 70b6dda309944660acf247e192b91913 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4dbb0679b1742808cff68ba15d0ee5b in term 1.
I20260923 23:18:07.129052 11000 raft_consensus.cc:2781] T 356cc4c8edaa40a685e7e2fd05c3b7c4 P b4dbb0679b1742808cff68ba15d0ee5b [term 1 FOLLOWER]: Leader election won for term 1
I20260923 23:18:07.129180 11000 raft_consensus.cc:686] T 356cc4c8edaa40a685e7e2fd05c3b7c4 P b4dbb0679b1742808cff68ba15d0ee5b [term 1 LEADER]: Becoming Leader. State: Replica: b4dbb0679b1742808cff68ba15d0ee5b, State: Running, Role: LEADER
I20260923 23:18:07.129258 11000 consensus_queue.cc:237] T 356cc4c8edaa40a685e7e2fd05c3b7c4 P b4dbb0679b1742808cff68ba15d0ee5b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8741179c89ea46c1b31edb6d1e63be3c" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 46693 } } peers { permanent_uuid: "70b6dda309944660acf247e192b91913" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 33277 } } peers { permanent_uuid: "b4dbb0679b1742808cff68ba15d0ee5b" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 34445 } }
I20260923 23:18:07.129997 10527 catalog_manager.cc:5495] T 356cc4c8edaa40a685e7e2fd05c3b7c4 P b4dbb0679b1742808cff68ba15d0ee5b reported cstate change: term changed from 0 to 1, leader changed from <none> to b4dbb0679b1742808cff68ba15d0ee5b (127.24.227.193). New cstate: current_term: 1 leader_uuid: "b4dbb0679b1742808cff68ba15d0ee5b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8741179c89ea46c1b31edb6d1e63be3c" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 46693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70b6dda309944660acf247e192b91913" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 33277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4dbb0679b1742808cff68ba15d0ee5b" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 34445 } health_report { overall_health: HEALTHY } } }
W20260923 23:18:07.135247 10849 tablet.cc:2369] T 356cc4c8edaa40a685e7e2fd05c3b7c4 P 70b6dda309944660acf247e192b91913: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 23:18:07.147215 10990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:07.145278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38736 (local address 127.24.227.193:34445)
0923 23:18:07.145312 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:07.145314 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:07.145338 (+ 24us) server_negotiation.cc:407] Connection header received
0923 23:18:07.145400 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:07.145400 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:07.145414 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:07.145472 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 23:18:07.145562 (+ 90us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:07.145998 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:07.146626 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:07.146786 (+ 160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:07.146837 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:07.146962 (+ 125us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 23:18:07.147042 (+ 80us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 23:18:07.147096 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:07.147135 (+ 39us) server_negotiation.cc:299] Negotiation successful
0923 23:18:07.147150 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260923 23:18:07.148849 10933 raft_consensus.cc:1261] T 356cc4c8edaa40a685e7e2fd05c3b7c4 P 8741179c89ea46c1b31edb6d1e63be3c [term 1 FOLLOWER]: Refusing update from remote peer b4dbb0679b1742808cff68ba15d0ee5b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 23:18:07.149130 10802 raft_consensus.cc:1261] T 356cc4c8edaa40a685e7e2fd05c3b7c4 P 70b6dda309944660acf247e192b91913 [term 1 FOLLOWER]: Refusing update from remote peer b4dbb0679b1742808cff68ba15d0ee5b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 23:18:07.149188 11000 consensus_queue.cc:1035] T 356cc4c8edaa40a685e7e2fd05c3b7c4 P b4dbb0679b1742808cff68ba15d0ee5b [LEADER]: Connected to new peer: Peer: permanent_uuid: "8741179c89ea46c1b31edb6d1e63be3c" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 46693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 23:18:07.149449 11000 consensus_queue.cc:1035] T 356cc4c8edaa40a685e7e2fd05c3b7c4 P b4dbb0679b1742808cff68ba15d0ee5b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70b6dda309944660acf247e192b91913" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 33277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 23:18:07.152221 11007 mvcc.cc:205] Tried to move back new op lower bound from 7332681675359203328 to 7332681675282034688. Current Snapshot: MvccSnapshot[applied={T|T < 7332681675359203328}]
I20260923 23:18:07.154284 10528 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49480:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 23:18:07.154393 10528 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49480:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 23:18:07.155061 10528 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2b39fbb7ccb04f518044ef8ea8fce44c: Sending DeleteTablet for 3 replicas of tablet 356cc4c8edaa40a685e7e2fd05c3b7c4
I20260923 23:18:07.155360 10913 tablet_service.cc:1514] Processing DeleteTablet for tablet 356cc4c8edaa40a685e7e2fd05c3b7c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 23:18:07 UTC) from {username='slave'} at 127.0.0.1:57944
I20260923 23:18:07.155369 10651 tablet_service.cc:1514] Processing DeleteTablet for tablet 356cc4c8edaa40a685e7e2fd05c3b7c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 23:18:07 UTC) from {username='slave'} at 127.0.0.1:38720
I20260923 23:18:07.155403 10782 tablet_service.cc:1514] Processing DeleteTablet for tablet 356cc4c8edaa40a685e7e2fd05c3b7c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 23:18:07 UTC) from {username='slave'} at 127.0.0.1:55392
I20260923 23:18:07.155799 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 10589
I20260923 23:18:07.156514 11018 tablet_replica.cc:330] T 356cc4c8edaa40a685e7e2fd05c3b7c4 P 70b6dda309944660acf247e192b91913: stopping tablet replica
I20260923 23:18:07.156600 11018 raft_consensus.cc:2229] T 356cc4c8edaa40a685e7e2fd05c3b7c4 P 70b6dda309944660acf247e192b91913 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 23:18:07.156665 11018 raft_consensus.cc:2258] T 356cc4c8edaa40a685e7e2fd05c3b7c4 P 70b6dda309944660acf247e192b91913 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 23:18:07.156858 11016 tablet_replica.cc:330] T 356cc4c8edaa40a685e7e2fd05c3b7c4 P 8741179c89ea46c1b31edb6d1e63be3c: stopping tablet replica
I20260923 23:18:07.156939 11016 raft_consensus.cc:2229] T 356cc4c8edaa40a685e7e2fd05c3b7c4 P 8741179c89ea46c1b31edb6d1e63be3c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 23:18:07.157022 11016 raft_consensus.cc:2258] T 356cc4c8edaa40a685e7e2fd05c3b7c4 P 8741179c89ea46c1b31edb6d1e63be3c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 23:18:07.157104 11018 ts_tablet_manager.cc:1905] T 356cc4c8edaa40a685e7e2fd05c3b7c4 P 70b6dda309944660acf247e192b91913: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 23:18:07.157518 11016 ts_tablet_manager.cc:1905] T 356cc4c8edaa40a685e7e2fd05c3b7c4 P 8741179c89ea46c1b31edb6d1e63be3c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 23:18:07.158682 11018 ts_tablet_manager.cc:1918] T 356cc4c8edaa40a685e7e2fd05c3b7c4 P 70b6dda309944660acf247e192b91913: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 23:18:07.158754 11018 log.cc:1196] T 356cc4c8edaa40a685e7e2fd05c3b7c4 P 70b6dda309944660acf247e192b91913: Deleting WAL directory at /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790205259943518-25487-0/minicluster-data/ts-1/wal/wals/356cc4c8edaa40a685e7e2fd05c3b7c4
I20260923 23:18:07.159047 11018 ts_tablet_manager.cc:1939] T 356cc4c8edaa40a685e7e2fd05c3b7c4 P 70b6dda309944660acf247e192b91913: Deleting consensus metadata
I20260923 23:18:07.159109 11016 ts_tablet_manager.cc:1918] T 356cc4c8edaa40a685e7e2fd05c3b7c4 P 8741179c89ea46c1b31edb6d1e63be3c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 23:18:07.159226 11016 log.cc:1196] T 356cc4c8edaa40a685e7e2fd05c3b7c4 P 8741179c89ea46c1b31edb6d1e63be3c: Deleting WAL directory at /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790205259943518-25487-0/minicluster-data/ts-2/wal/wals/356cc4c8edaa40a685e7e2fd05c3b7c4
I20260923 23:18:07.159512 11016 ts_tablet_manager.cc:1939] T 356cc4c8edaa40a685e7e2fd05c3b7c4 P 8741179c89ea46c1b31edb6d1e63be3c: Deleting consensus metadata
I20260923 23:18:07.159512 10515 catalog_manager.cc:4841] TS 70b6dda309944660acf247e192b91913 (127.24.227.194:33277): tablet 356cc4c8edaa40a685e7e2fd05c3b7c4 (table test-table [id=6e39d830e6c54fbbaf6e957d82e43270]) successfully deleted
I20260923 23:18:07.159916 10515 catalog_manager.cc:4841] TS 8741179c89ea46c1b31edb6d1e63be3c (127.24.227.195:46693): tablet 356cc4c8edaa40a685e7e2fd05c3b7c4 (table test-table [id=6e39d830e6c54fbbaf6e957d82e43270]) successfully deleted
W20260923 23:18:07.161456 10516 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.227.193:34445 (error 108)
I20260923 23:18:07.161674 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 10720
I20260923 23:18:07.162278 10991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:07.161758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.193:34445 (local address 127.0.0.1:38752)
0923 23:18:07.162206 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:07.162254 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.193:34445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":427}
W20260923 23:18:07.162405 10516 catalog_manager.cc:4584] TS b4dbb0679b1742808cff68ba15d0ee5b (127.24.227.193:34445): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 356cc4c8edaa40a685e7e2fd05c3b7c4: Network error: Client connection negotiation failed: client connection to 127.24.227.193:34445: connect: Connection refused (error 111)
I20260923 23:18:07.167251 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 10851
I20260923 23:18:07.171474 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 10498
2026-09-23T23:18:07Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (714 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790205259943518-25487-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 23:18:07 dist-test-slave-754n krb5kdc[11022](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 23:18:07 dist-test-slave-754n krb5kdc[11022](info): set up 2 sockets
Sep 23 23:18:07 dist-test-slave-754n krb5kdc[11022](info): commencing operation
krb5kdc: starting...
W20260923 23:18:08.183645 25487 mini_kdc.cc:121] Time spent starting KDC: real 0.993s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790205259943518-25487-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790205259943518-25487-0/krb5kdc/test-user.keytab.
Sep 23 23:18:08 dist-test-slave-754n krb5kdc[11022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205488, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T23:18:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T23:18:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.227.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.227.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.227.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab.
Entry for principal kudu/127.24.227.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab.
I20260923 23:18:08.292183 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790205259943518-25487-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790205259943518-25487-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790205259943518-25487-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.254:34395
--webserver_interface=127.24.227.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab
--principal=kudu/127.24.227.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:33099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.227.254:34395
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:18:08.367478 11035 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:18:08.367657 11035 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:18:08.367697 11035 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:18:08.368737 11035 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 23:18:08.368775 11035 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:18:08.368789 11035 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:18:08.368801 11035 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 23:18:08.368813 11035 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 23:18:08.370102 11035 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:33099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790205259943518-25487-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790205259943518-25487-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.227.254:34395
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab
--principal=kudu/127.24.227.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.254:34395
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.227.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11035
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790205259943518-25487-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:18:08.370301 11035 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:18:08.370450 11035 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:18:08.375020 11043 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:18:08.375025 11041 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:18:08.375252 11035 server_base.cc:1034] running on GCE node
W20260923 23:18:08.375046 11040 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:18:08.375561 11035 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:18:08.375818 11035 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:18:08.376997 11035 hybrid_clock.cc:648] HybridClock initialized: now 1790205488376976 us; error 37 us; skew 500 ppm
Sep 23 23:18:08 dist-test-slave-754n krb5kdc[11022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205488, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:18:08.379336 11035 init.cc:377] Logged in from keytab as kudu/127.24.227.254@KRBTEST.COM (short username kudu)
I20260923 23:18:08.379776 11035 webserver.cc:460] Webserver started at http://127.24.227.254:42229/ using document root <none> and password file <none>
I20260923 23:18:08.380048 11035 fs_manager.cc:359] Metadata directory not provided
I20260923 23:18:08.380113 11035 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:18:08.380255 11035 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:18:08.381261 11035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790205259943518-25487-0/minicluster-data/master-0/data/instance:
uuid: "f65fe1a63de64b22ad7c872564d22e1c"
format_stamp: "Formatted at 2026-09-23 23:18:08 on dist-test-slave-754n"
server_key: "2c4ce37c04d10f141027e265b9291b3a"
server_key_iv: "9f2ba67c6528442cb4a04e430ccb007e"
server_key_version: "encryptionkey@0"
I20260923 23:18:08.381604 11035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790205259943518-25487-0/minicluster-data/master-0/wal/instance:
uuid: "f65fe1a63de64b22ad7c872564d22e1c"
format_stamp: "Formatted at 2026-09-23 23:18:08 on dist-test-slave-754n"
server_key: "2c4ce37c04d10f141027e265b9291b3a"
server_key_iv: "9f2ba67c6528442cb4a04e430ccb007e"
server_key_version: "encryptionkey@0"
I20260923 23:18:08.382851 11035 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 23:18:08.383497 11050 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:18:08.383682 11035 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 23:18:08.383783 11035 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790205259943518-25487-0/minicluster-data/master-0/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790205259943518-25487-0/minicluster-data/master-0/wal
uuid: "f65fe1a63de64b22ad7c872564d22e1c"
format_stamp: "Formatted at 2026-09-23 23:18:08 on dist-test-slave-754n"
server_key: "2c4ce37c04d10f141027e265b9291b3a"
server_key_iv: "9f2ba67c6528442cb4a04e430ccb007e"
server_key_version: "encryptionkey@0"
I20260923 23:18:08.383864 11035 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790205259943518-25487-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790205259943518-25487-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790205259943518-25487-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:18:08.399015 11035 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:18:08.401297 11035 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:18:08.401453 11035 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:18:08.405148 11035 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.254:34395
I20260923 23:18:08.405179 11102 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.254:34395 every 8 connection(s)
I20260923 23:18:08.405490 11035 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
I20260923 23:18:08.405933 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 11035
I20260923 23:18:08.406054 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790205259943518-25487-0/minicluster-data/master-0/wal/instance
I20260923 23:18:08.406250 25487 external_mini_cluster.cc:1428] Setting key 0666c9562efb253e3a0dc84f93033110
I20260923 23:18:08.406486 11103 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:18:08.409096 11103 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f65fe1a63de64b22ad7c872564d22e1c: Bootstrap starting.
I20260923 23:18:08.409667 11103 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f65fe1a63de64b22ad7c872564d22e1c: Neither blocks nor log segments found. Creating new log.
I20260923 23:18:08.409950 11103 log.cc:824] T 00000000000000000000000000000000 P f65fe1a63de64b22ad7c872564d22e1c: Log is configured to *not* fsync() on all Append() calls
I20260923 23:18:08.410612 11103 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f65fe1a63de64b22ad7c872564d22e1c: No bootstrap required, opened a new log
Sep 23 23:18:08 dist-test-slave-754n krb5kdc[11022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205488, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
I20260923 23:18:08.412588 11103 raft_consensus.cc:348] T 00000000000000000000000000000000 P f65fe1a63de64b22ad7c872564d22e1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f65fe1a63de64b22ad7c872564d22e1c" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 34395 } }
I20260923 23:18:08.412732 11103 raft_consensus.cc:374] T 00000000000000000000000000000000 P f65fe1a63de64b22ad7c872564d22e1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:18:08.412781 11103 raft_consensus.cc:729] T 00000000000000000000000000000000 P f65fe1a63de64b22ad7c872564d22e1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f65fe1a63de64b22ad7c872564d22e1c, State: Initialized, Role: FOLLOWER
I20260923 23:18:08.412925 11103 consensus_queue.cc:260] T 00000000000000000000000000000000 P f65fe1a63de64b22ad7c872564d22e1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f65fe1a63de64b22ad7c872564d22e1c" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 34395 } }
I20260923 23:18:08.413033 11103 raft_consensus.cc:388] T 00000000000000000000000000000000 P f65fe1a63de64b22ad7c872564d22e1c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 23:18:08.413067 11103 raft_consensus.cc:482] T 00000000000000000000000000000000 P f65fe1a63de64b22ad7c872564d22e1c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 23:18:08.413118 11103 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f65fe1a63de64b22ad7c872564d22e1c [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:18:08.413707 11103 raft_consensus.cc:504] T 00000000000000000000000000000000 P f65fe1a63de64b22ad7c872564d22e1c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f65fe1a63de64b22ad7c872564d22e1c" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 34395 } }
I20260923 23:18:08.413841 11103 leader_election.cc:302] T 00000000000000000000000000000000 P f65fe1a63de64b22ad7c872564d22e1c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f65fe1a63de64b22ad7c872564d22e1c; no voters:
I20260923 23:18:08.414029 11103 leader_election.cc:288] T 00000000000000000000000000000000 P f65fe1a63de64b22ad7c872564d22e1c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 23:18:08.414073 11108 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f65fe1a63de64b22ad7c872564d22e1c [term 1 FOLLOWER]: Leader election won for term 1
I20260923 23:18:08.414202 11108 raft_consensus.cc:686] T 00000000000000000000000000000000 P f65fe1a63de64b22ad7c872564d22e1c [term 1 LEADER]: Becoming Leader. State: Replica: f65fe1a63de64b22ad7c872564d22e1c, State: Running, Role: LEADER
I20260923 23:18:08.414285 11103 sys_catalog.cc:564] T 00000000000000000000000000000000 P f65fe1a63de64b22ad7c872564d22e1c [sys.catalog]: configured and running, proceeding with master startup.
I20260923 23:18:08.414342 11108 consensus_queue.cc:237] T 00000000000000000000000000000000 P f65fe1a63de64b22ad7c872564d22e1c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f65fe1a63de64b22ad7c872564d22e1c" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 34395 } }
I20260923 23:18:08.414710 11108 sys_catalog.cc:455] T 00000000000000000000000000000000 P f65fe1a63de64b22ad7c872564d22e1c [sys.catalog]: SysCatalogTable state changed. Reason: New leader f65fe1a63de64b22ad7c872564d22e1c. Latest consensus state: current_term: 1 leader_uuid: "f65fe1a63de64b22ad7c872564d22e1c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f65fe1a63de64b22ad7c872564d22e1c" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 34395 } } }
I20260923 23:18:08.414878 11108 sys_catalog.cc:458] T 00000000000000000000000000000000 P f65fe1a63de64b22ad7c872564d22e1c [sys.catalog]: This master's current role is: LEADER
I20260923 23:18:08.415030 11109 sys_catalog.cc:455] T 00000000000000000000000000000000 P f65fe1a63de64b22ad7c872564d22e1c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f65fe1a63de64b22ad7c872564d22e1c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f65fe1a63de64b22ad7c872564d22e1c" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 34395 } } }
I20260923 23:18:08.415102 11109 sys_catalog.cc:458] T 00000000000000000000000000000000 P f65fe1a63de64b22ad7c872564d22e1c [sys.catalog]: This master's current role is: LEADER
I20260923 23:18:08.415117 11117 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 23:18:08.415614 11117 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 23:18:08.416618 11106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:08.406684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56292 (local address 127.24.227.254:34395)
0923 23:18:08.406860 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:08.406869 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:08.406898 (+ 29us) server_negotiation.cc:407] Connection header received
0923 23:18:08.407478 (+ 580us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:08.407484 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:08.407528 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:08.407671 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:08.408327 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:08.409168 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:08.409942 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:08.410413 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:08.410525 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:08.412751 (+ 2226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:18:08.412755 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:18:08.412755 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:18:08.412767 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:18:08.414629 (+ 1862us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:08.414991 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:08.414994 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:08.414994 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:08.415089 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:08.415326 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:08.415327 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:08.415327 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:08.415599 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:18:08.415754 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:08.416296 (+ 542us) server_negotiation.cc:299] Negotiation successful
0923 23:18:08.416447 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":89,"threads_started":1}
I20260923 23:18:08.417409 11117 catalog_manager.cc:1347] Generated new cluster ID: 88d87f352ef240cf9dbaf5c9b8727069
I20260923 23:18:08.417461 11117 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 23:18:08.430001 11117 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 23:18:08.430496 11117 catalog_manager.cc:1504] Loading token signing keys...
I20260923 23:18:08.437707 11117 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f65fe1a63de64b22ad7c872564d22e1c: Generated new TSK 0
I20260923 23:18:08.437873 11117 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.227.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.227.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.227.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.193.keytab.
Entry for principal kudu/127.24.227.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.193.keytab.
I20260923 23:18:08.470353 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790205259943518-25487-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790205259943518-25487-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790205259943518-25487-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790205259943518-25487-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.227.193:0
--local_ip_for_outbound_sockets=127.24.227.193
--webserver_interface=127.24.227.193
--webserver_port=0
--tserver_master_addrs=127.24.227.254:34395
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.193.keytab
--principal=kudu/127.24.227.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:33099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:18:08.557657 11129 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:18:08.557847 11129 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:18:08.557868 11129 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:18:08.558952 11129 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:18:08.558990 11129 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:18:08.559039 11129 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.227.193
I20260923 23:18:08.560446 11129 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:33099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790205259943518-25487-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790205259943518-25487-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.193.keytab
--principal=kudu/127.24.227.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.227.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790205259943518-25487-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.227.193
--webserver_port=0
--tserver_master_addrs=127.24.227.254:34395
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11129
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.227.193
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790205259943518-25487-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:18:08.560665 11129 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:18:08.560897 11129 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:18:08.565531 11137 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:18:08.565531 11135 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:18:08.565557 11134 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:18:08.565744 11129 server_base.cc:1034] running on GCE node
I20260923 23:18:08.565996 11129 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:18:08.566250 11129 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:18:08.567415 11129 hybrid_clock.cc:648] HybridClock initialized: now 1790205488567394 us; error 37 us; skew 500 ppm
Sep 23 23:18:08 dist-test-slave-754n krb5kdc[11022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205488, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:18:08.569911 11129 init.cc:377] Logged in from keytab as kudu/127.24.227.193@KRBTEST.COM (short username kudu)
I20260923 23:18:08.570335 11129 webserver.cc:460] Webserver started at http://127.24.227.193:33429/ using document root <none> and password file <none>
I20260923 23:18:08.570499 11129 fs_manager.cc:359] Metadata directory not provided
I20260923 23:18:08.570536 11129 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:18:08.570617 11129 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:18:08.571501 11129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790205259943518-25487-0/minicluster-data/ts-0/data/instance:
uuid: "0deb4a30e6bc41c39bd9c6cab1925f80"
format_stamp: "Formatted at 2026-09-23 23:18:08 on dist-test-slave-754n"
server_key: "55731172d6820cb793a0151a0f210bd9"
server_key_iv: "58a9d0414521e297dbfc65d2726e3691"
server_key_version: "encryptionkey@0"
I20260923 23:18:08.571784 11129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790205259943518-25487-0/minicluster-data/ts-0/wal/instance:
uuid: "0deb4a30e6bc41c39bd9c6cab1925f80"
format_stamp: "Formatted at 2026-09-23 23:18:08 on dist-test-slave-754n"
server_key: "55731172d6820cb793a0151a0f210bd9"
server_key_iv: "58a9d0414521e297dbfc65d2726e3691"
server_key_version: "encryptionkey@0"
I20260923 23:18:08.573014 11129 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 23:18:08.573629 11144 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:18:08.573807 11129 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 23:18:08.573868 11129 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790205259943518-25487-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790205259943518-25487-0/minicluster-data/ts-0/wal
uuid: "0deb4a30e6bc41c39bd9c6cab1925f80"
format_stamp: "Formatted at 2026-09-23 23:18:08 on dist-test-slave-754n"
server_key: "55731172d6820cb793a0151a0f210bd9"
server_key_iv: "58a9d0414521e297dbfc65d2726e3691"
server_key_version: "encryptionkey@0"
I20260923 23:18:08.573915 11129 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790205259943518-25487-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790205259943518-25487-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790205259943518-25487-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:18:08.597443 11129 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:18:08.599830 11129 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:18:08.600031 11129 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:18:08.600234 11129 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 23:18:08.600517 11129 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 23:18:08.600549 11129 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 23:18:08.600570 11129 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 23:18:08.600616 11129 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 23:18:08.606102 11129 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.193:43587
I20260923 23:18:08.606166 11257 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.193:43587 every 8 connection(s)
I20260923 23:18:08.606475 11129 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790205259943518-25487-0/minicluster-data/ts-0/data/info.pb
Sep 23 23:18:08 dist-test-slave-754n krb5kdc[11022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205488, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.193@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
I20260923 23:18:08.614643 11106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:08.607131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.193:39793 (local address 127.24.227.254:34395)
0923 23:18:08.607174 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:08.607176 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:08.607344 (+ 168us) server_negotiation.cc:407] Connection header received
0923 23:18:08.607795 (+ 451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:08.607796 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:08.607804 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:08.607859 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:08.608747 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:08.609245 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:08.610134 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:08.610344 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:08.610399 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:08.612328 (+ 1929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:18:08.612329 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:18:08.612329 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:18:08.612335 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:18:08.613480 (+ 1145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:08.613838 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:08.613838 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:08.613838 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:08.613887 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:08.614117 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:08.614117 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:08.614118 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:08.614262 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:18:08.614318 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:08.614524 (+ 206us) server_negotiation.cc:299] Negotiation successful
0923 23:18:08.614584 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 23:18:08.614782 11260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:08.607099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:34395 (local address 127.24.227.193:39793)
0923 23:18:08.607238 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:08.607276 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0923 23:18:08.607704 (+ 428us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:18:08.607961 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:18:08.607964 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:18:08.608459 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:18:08.608664 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:08.608669 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:08.609320 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:08.609321 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:08.610060 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:08.610061 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:08.610414 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:08.610415 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:08.610505 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:08.610961 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:18:08.610968 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:18:08.612253 (+ 1285us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:18:08.613556 (+ 1303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:18:08.613562 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:18:08.613563 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:18:08.613774 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:18:08.613958 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:18:08.613959 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:18:08.613959 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:18:08.614061 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:18:08.614313 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:18:08.614318 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:18:08.614418 (+ 100us) client_negotiation.cc:769] Sending connection context
0923 23:18:08.614497 (+ 79us) client_negotiation.cc:240] Negotiation successful
0923 23:18:08.614601 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":77,"threads_started":1}
I20260923 23:18:08.614941 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 11129
I20260923 23:18:08.615069 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790205259943518-25487-0/minicluster-data/ts-0/wal/instance
I20260923 23:18:08.615284 25487 external_mini_cluster.cc:1428] Setting key 7f593b58fca8269db98a3f30250b21f3
I20260923 23:18:08.615398 11258 heartbeater.cc:347] Connected to a master server at 127.24.227.254:34395
I20260923 23:18:08.615489 11258 heartbeater.cc:464] Registering TS with master...
I20260923 23:18:08.615660 11258 heartbeater.cc:511] Master 127.24.227.254:34395 requested a full tablet report, sending...
I20260923 23:18:08.616334 11067 ts_manager.cc:194] Registered new tserver with Master: 0deb4a30e6bc41c39bd9c6cab1925f80 (127.24.227.193:43587)
I20260923 23:18:08.617197 11067 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.227.193@KRBTEST.COM'} at 127.24.227.193:39793
WARNING: no policy specified for kudu/127.24.227.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.227.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.227.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.194.keytab.
Entry for principal kudu/127.24.227.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.194.keytab.
I20260923 23:18:08.638636 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790205259943518-25487-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790205259943518-25487-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790205259943518-25487-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790205259943518-25487-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.227.194:0
--local_ip_for_outbound_sockets=127.24.227.194
--webserver_interface=127.24.227.194
--webserver_port=0
--tserver_master_addrs=127.24.227.254:34395
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.194.keytab
--principal=kudu/127.24.227.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:33099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:18:08.715651 11263 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:18:08.715833 11263 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:18:08.715857 11263 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:18:08.716935 11263 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:18:08.716967 11263 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:18:08.716993 11263 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.227.194
I20260923 23:18:08.718287 11263 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:33099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790205259943518-25487-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790205259943518-25487-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.194.keytab
--principal=kudu/127.24.227.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.227.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790205259943518-25487-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.227.194
--webserver_port=0
--tserver_master_addrs=127.24.227.254:34395
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11263
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.227.194
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790205259943518-25487-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:18:08.718454 11263 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:18:08.718603 11263 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:18:08.723104 11271 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:18:08.723109 11268 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:18:08.723165 11269 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:18:08.723295 11263 server_base.cc:1034] running on GCE node
I20260923 23:18:08.723482 11263 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:18:08.723738 11263 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:18:08.724898 11263 hybrid_clock.cc:648] HybridClock initialized: now 1790205488724892 us; error 37 us; skew 500 ppm
Sep 23 23:18:08 dist-test-slave-754n krb5kdc[11022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205488, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:18:08.727286 11263 init.cc:377] Logged in from keytab as kudu/127.24.227.194@KRBTEST.COM (short username kudu)
I20260923 23:18:08.727684 11263 webserver.cc:460] Webserver started at http://127.24.227.194:33185/ using document root <none> and password file <none>
I20260923 23:18:08.727850 11263 fs_manager.cc:359] Metadata directory not provided
I20260923 23:18:08.727947 11263 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:18:08.728034 11263 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:18:08.728910 11263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790205259943518-25487-0/minicluster-data/ts-1/data/instance:
uuid: "3e258479bc65400c995e200ec210d27b"
format_stamp: "Formatted at 2026-09-23 23:18:08 on dist-test-slave-754n"
server_key: "ec61fd28d91d372c590609eecc51d27b"
server_key_iv: "81eaee7941fec5a0850f15190ff1b4ef"
server_key_version: "encryptionkey@0"
I20260923 23:18:08.729182 11263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790205259943518-25487-0/minicluster-data/ts-1/wal/instance:
uuid: "3e258479bc65400c995e200ec210d27b"
format_stamp: "Formatted at 2026-09-23 23:18:08 on dist-test-slave-754n"
server_key: "ec61fd28d91d372c590609eecc51d27b"
server_key_iv: "81eaee7941fec5a0850f15190ff1b4ef"
server_key_version: "encryptionkey@0"
I20260923 23:18:08.730360 11263 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 23:18:08.730975 11278 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:18:08.731127 11263 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 23:18:08.731202 11263 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790205259943518-25487-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790205259943518-25487-0/minicluster-data/ts-1/wal
uuid: "3e258479bc65400c995e200ec210d27b"
format_stamp: "Formatted at 2026-09-23 23:18:08 on dist-test-slave-754n"
server_key: "ec61fd28d91d372c590609eecc51d27b"
server_key_iv: "81eaee7941fec5a0850f15190ff1b4ef"
server_key_version: "encryptionkey@0"
I20260923 23:18:08.731263 11263 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790205259943518-25487-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790205259943518-25487-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790205259943518-25487-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:18:08.739262 11263 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:18:08.740931 11263 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:18:08.741037 11263 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:18:08.741189 11263 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 23:18:08.741416 11263 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 23:18:08.741446 11263 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 23:18:08.741464 11263 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 23:18:08.741477 11263 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 23:18:08.746994 11263 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.194:36153
I20260923 23:18:08.747074 11391 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.194:36153 every 8 connection(s)
I20260923 23:18:08.747395 11263 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790205259943518-25487-0/minicluster-data/ts-1/data/info.pb
I20260923 23:18:08.752192 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 11263
I20260923 23:18:08.752332 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790205259943518-25487-0/minicluster-data/ts-1/wal/instance
I20260923 23:18:08.752514 25487 external_mini_cluster.cc:1428] Setting key c64bd702f3371d06732c23c4e67bf851
Sep 23 23:18:08 dist-test-slave-754n krb5kdc[11022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205488, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.194@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
I20260923 23:18:08.757421 11394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:08.748115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:34395 (local address 127.24.227.194:38433)
0923 23:18:08.748286 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:08.748318 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0923 23:18:08.748770 (+ 452us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:18:08.749011 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:18:08.749013 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:18:08.749567 (+ 554us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:18:08.749755 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:08.749759 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:08.750800 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:08.750801 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:08.751589 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:08.751591 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:08.752137 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:08.752138 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:08.752245 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:08.752877 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:18:08.752886 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:18:08.754490 (+ 1604us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:18:08.756133 (+ 1643us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:18:08.756140 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:18:08.756142 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:18:08.756396 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:18:08.756629 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:18:08.756629 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:18:08.756629 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:18:08.756722 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:18:08.756995 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:18:08.756997 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:18:08.757094 (+ 97us) client_negotiation.cc:769] Sending connection context
0923 23:18:08.757165 (+ 71us) client_negotiation.cc:240] Negotiation successful
0923 23:18:08.757280 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":96,"threads_started":1}
I20260923 23:18:08.757443 11106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:08.748137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.194:38433 (local address 127.24.227.254:34395)
0923 23:18:08.748198 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:08.748200 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:08.748398 (+ 198us) server_negotiation.cc:407] Connection header received
0923 23:18:08.748861 (+ 463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:08.748863 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:08.748875 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:08.748950 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:08.749845 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:08.750703 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:08.751683 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:08.752050 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:08.752131 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:08.754619 (+ 2488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:18:08.754620 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:18:08.754620 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:18:08.754627 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:18:08.756047 (+ 1420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:08.756486 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:08.756488 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:08.756488 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:08.756553 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:08.756786 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:08.756787 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:08.756788 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:08.756927 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:18:08.757009 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:08.757217 (+ 208us) server_negotiation.cc:299] Negotiation successful
0923 23:18:08.757302 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260923 23:18:08.758075 11392 heartbeater.cc:347] Connected to a master server at 127.24.227.254:34395
I20260923 23:18:08.758172 11392 heartbeater.cc:464] Registering TS with master...
I20260923 23:18:08.758364 11392 heartbeater.cc:511] Master 127.24.227.254:34395 requested a full tablet report, sending...
I20260923 23:18:08.758733 11067 ts_manager.cc:194] Registered new tserver with Master: 3e258479bc65400c995e200ec210d27b (127.24.227.194:36153)
I20260923 23:18:08.759106 11067 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.227.194@KRBTEST.COM'} at 127.24.227.194:38433
WARNING: no policy specified for kudu/127.24.227.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.227.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.227.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.195.keytab.
Entry for principal kudu/127.24.227.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.195.keytab.
I20260923 23:18:08.777168 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790205259943518-25487-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790205259943518-25487-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790205259943518-25487-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790205259943518-25487-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.227.195:0
--local_ip_for_outbound_sockets=127.24.227.195
--webserver_interface=127.24.227.195
--webserver_port=0
--tserver_master_addrs=127.24.227.254:34395
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.195.keytab
--principal=kudu/127.24.227.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:33099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:18:08.864759 11397 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:18:08.864943 11397 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:18:08.864965 11397 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:18:08.866025 11397 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:18:08.866061 11397 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:18:08.866086 11397 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.227.195
I20260923 23:18:08.867381 11397 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:33099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790205259943518-25487-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790205259943518-25487-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.195.keytab
--principal=kudu/127.24.227.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.227.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790205259943518-25487-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.227.195
--webserver_port=0
--tserver_master_addrs=127.24.227.254:34395
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11397
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.227.195
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790205259943518-25487-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:18:08.867534 11397 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:18:08.867682 11397 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:18:08.872278 11405 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:18:08.872349 11402 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:18:08.872344 11397 server_base.cc:1034] running on GCE node
W20260923 23:18:08.872279 11403 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:18:08.872668 11397 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:18:08.872898 11397 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:18:08.874049 11397 hybrid_clock.cc:648] HybridClock initialized: now 1790205488874038 us; error 27 us; skew 500 ppm
Sep 23 23:18:08 dist-test-slave-754n krb5kdc[11022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205488, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:18:08.876477 11397 init.cc:377] Logged in from keytab as kudu/127.24.227.195@KRBTEST.COM (short username kudu)
I20260923 23:18:08.876928 11397 webserver.cc:460] Webserver started at http://127.24.227.195:45143/ using document root <none> and password file <none>
I20260923 23:18:08.877154 11397 fs_manager.cc:359] Metadata directory not provided
I20260923 23:18:08.877224 11397 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:18:08.877350 11397 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:18:08.878315 11397 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790205259943518-25487-0/minicluster-data/ts-2/data/instance:
uuid: "6c447d4929ea488e81b6897b1d959580"
format_stamp: "Formatted at 2026-09-23 23:18:08 on dist-test-slave-754n"
server_key: "c7d563a7909d3b07c17fb068fc33ffed"
server_key_iv: "4b9c3030f92d7198eaada5cd348fc475"
server_key_version: "encryptionkey@0"
I20260923 23:18:08.878651 11397 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790205259943518-25487-0/minicluster-data/ts-2/wal/instance:
uuid: "6c447d4929ea488e81b6897b1d959580"
format_stamp: "Formatted at 2026-09-23 23:18:08 on dist-test-slave-754n"
server_key: "c7d563a7909d3b07c17fb068fc33ffed"
server_key_iv: "4b9c3030f92d7198eaada5cd348fc475"
server_key_version: "encryptionkey@0"
I20260923 23:18:08.879824 11397 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260923 23:18:08.880519 11412 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:18:08.880690 11397 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 23:18:08.880784 11397 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790205259943518-25487-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790205259943518-25487-0/minicluster-data/ts-2/wal
uuid: "6c447d4929ea488e81b6897b1d959580"
format_stamp: "Formatted at 2026-09-23 23:18:08 on dist-test-slave-754n"
server_key: "c7d563a7909d3b07c17fb068fc33ffed"
server_key_iv: "4b9c3030f92d7198eaada5cd348fc475"
server_key_version: "encryptionkey@0"
I20260923 23:18:08.880867 11397 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790205259943518-25487-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790205259943518-25487-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790205259943518-25487-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:18:08.902802 11397 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:18:08.904948 11397 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:18:08.905107 11397 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:18:08.905344 11397 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 23:18:08.905620 11397 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 23:18:08.905671 11397 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 23:18:08.905717 11397 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 23:18:08.905745 11397 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 23:18:08.911568 11397 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.195:42475
I20260923 23:18:08.911602 11525 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.195:42475 every 8 connection(s)
I20260923 23:18:08.911912 11397 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790205259943518-25487-0/minicluster-data/ts-2/data/info.pb
I20260923 23:18:08.912281 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 11397
I20260923 23:18:08.912403 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790205259943518-25487-0/minicluster-data/ts-2/wal/instance
I20260923 23:18:08.912595 25487 external_mini_cluster.cc:1428] Setting key edff498dbab7112deb559a42d619d5c7
Sep 23 23:18:08 dist-test-slave-754n krb5kdc[11022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205488, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.195@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
I20260923 23:18:08.920730 11106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:08.912618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.195:48633 (local address 127.24.227.254:34395)
0923 23:18:08.912667 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:08.912669 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:08.912901 (+ 232us) server_negotiation.cc:407] Connection header received
0923 23:18:08.913600 (+ 699us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:08.913601 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:08.913630 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:08.913693 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:08.914634 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:08.915271 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:08.916364 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:08.916631 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:08.916685 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:08.918438 (+ 1753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:18:08.918439 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:18:08.918439 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:18:08.918445 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:18:08.919595 (+ 1150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:08.920004 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:08.920005 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:08.920005 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:08.920055 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:08.920270 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:08.920270 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:08.920270 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:08.920365 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:18:08.920414 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:08.920606 (+ 192us) server_negotiation.cc:299] Negotiation successful
0923 23:18:08.920659 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260923 23:18:08.920866 11528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:08.912609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:34395 (local address 127.24.227.195:48633)
0923 23:18:08.912766 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:08.912812 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0923 23:18:08.913480 (+ 668us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:18:08.913760 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:18:08.913763 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:18:08.914343 (+ 580us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:18:08.914519 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:08.914523 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:08.915352 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:08.915353 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:08.916263 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:08.916265 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:08.916699 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:08.916700 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:08.916790 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:08.917140 (+ 350us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:18:08.917147 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:18:08.918345 (+ 1198us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:18:08.919666 (+ 1321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:18:08.919671 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:18:08.919672 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:18:08.919943 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:18:08.920115 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:18:08.920116 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:18:08.920116 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:18:08.920205 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:18:08.920418 (+ 213us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:18:08.920421 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:18:08.920519 (+ 98us) client_negotiation.cc:769] Sending connection context
0923 23:18:08.920593 (+ 74us) client_negotiation.cc:240] Negotiation successful
0923 23:18:08.920696 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":85,"threads_started":1}
I20260923 23:18:08.921305 11526 heartbeater.cc:347] Connected to a master server at 127.24.227.254:34395
I20260923 23:18:08.921376 11526 heartbeater.cc:464] Registering TS with master...
I20260923 23:18:08.921559 11526 heartbeater.cc:511] Master 127.24.227.254:34395 requested a full tablet report, sending...
I20260923 23:18:08.921906 11067 ts_manager.cc:194] Registered new tserver with Master: 6c447d4929ea488e81b6897b1d959580 (127.24.227.195:42475)
I20260923 23:18:08.922241 11067 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.227.195@KRBTEST.COM'} at 127.24.227.195:48633
I20260923 23:18:08.924273 25487 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 23:18:08.930608 11106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:08.925563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35332 (local address 127.24.227.254:34395)
0923 23:18:08.925593 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:08.925595 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:08.925695 (+ 100us) server_negotiation.cc:407] Connection header received
0923 23:18:08.925731 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:08.925732 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:08.925739 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:08.925776 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:08.926199 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:08.926589 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:08.927183 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:08.927336 (+ 153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:08.927379 (+ 43us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:08.928454 (+ 1075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:18:08.928456 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:18:08.928456 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:18:08.928461 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:18:08.929690 (+ 1229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:08.929998 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:08.929999 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:08.929999 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:08.930061 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:08.930255 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:08.930256 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:08.930256 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:08.930357 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:18:08.930427 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:08.930510 (+ 83us) server_negotiation.cc:299] Negotiation successful
0923 23:18:08.930555 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260923 23:18:08.932030 11067 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35332:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 23:18:08.933815 11067 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 23:18:08.939930 11536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:08.935824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45336 (local address 127.24.227.195:42475)
0923 23:18:08.936570 (+ 746us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:08.936574 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:08.936582 (+ 8us) server_negotiation.cc:407] Connection header received
0923 23:18:08.936612 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:08.936614 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:08.936628 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:08.936704 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:18:08.936819 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:08.937503 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:08.938837 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:08.939448 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:08.939551 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:08.939602 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:08.939793 (+ 191us) server_negotiation.cc:299] Negotiation successful
0923 23:18:08.939829 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":719,"thread_start_us":93,"threads_started":1}
I20260923 23:18:08.939919 11537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:08.935753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.195:42475 (local address 127.0.0.1:45336)
0923 23:18:08.936042 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:08.936052 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 23:18:08.936118 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:18:08.936702 (+ 584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:18:08.936702 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:18:08.936704 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:18:08.936746 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:08.936748 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:08.937615 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:08.937615 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:08.938763 (+ 1148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:08.938765 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:08.939549 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:08.939550 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:08.939629 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:08.939655 (+ 26us) client_negotiation.cc:769] Sending connection context
0923 23:18:08.939776 (+ 121us) client_negotiation.cc:240] Negotiation successful
0923 23:18:08.939828 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"mutex_wait_us":2,"thread_start_us":53,"threads_started":1}
I20260923 23:18:08.940434 11535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:08.935945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.193:43587 (local address 127.0.0.1:53284)
0923 23:18:08.936410 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:08.936420 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 23:18:08.936471 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:18:08.936716 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:18:08.936718 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:18:08.936720 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:18:08.936780 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:08.936783 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:08.938143 (+ 1360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:08.938144 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:08.939087 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:08.939089 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:08.940127 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:08.940128 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:08.940231 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:08.940235 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 23:18:08.940288 (+ 53us) client_negotiation.cc:240] Negotiation successful
0923 23:18:08.940341 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":453,"mutex_wait_us":14,"thread_start_us":55,"threads_started":1}
I20260923 23:18:08.940533 11538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:08.935957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55632 (local address 127.24.227.194:36153)
0923 23:18:08.936139 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:08.936143 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:08.936152 (+ 9us) server_negotiation.cc:407] Connection header received
0923 23:18:08.936188 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:08.936189 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:08.936240 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:08.936326 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:18:08.936500 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:08.937889 (+ 1389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:08.939056 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:08.939828 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:08.939904 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:08.940121 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:08.940400 (+ 279us) server_negotiation.cc:299] Negotiation successful
0923 23:18:08.940444 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":71,"threads_started":1}
I20260923 23:18:08.940672 11539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:08.936131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53284 (local address 127.24.227.193:43587)
0923 23:18:08.936329 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:08.936333 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:08.936500 (+ 167us) server_negotiation.cc:407] Connection header received
0923 23:18:08.936543 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:08.936545 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:08.936560 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:08.936633 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:18:08.936870 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:08.937959 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:08.939168 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:08.939967 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:08.940038 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:08.940092 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:08.940533 (+ 441us) server_negotiation.cc:299] Negotiation successful
0923 23:18:08.940578 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":135,"threads_started":1}
I20260923 23:18:08.940699 11534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:08.935753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.194:36153 (local address 127.0.0.1:55632)
0923 23:18:08.935909 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:08.935942 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0923 23:18:08.935998 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:18:08.936332 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:18:08.936334 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:18:08.936337 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:18:08.936405 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:08.936406 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:08.937992 (+ 1586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:08.937993 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:08.938973 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:08.938974 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:08.940148 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:08.940149 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:08.940243 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:08.940246 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 23:18:08.940292 (+ 46us) client_negotiation.cc:240] Negotiation successful
0923 23:18:08.940342 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":47,"threads_started":1}
I20260923 23:18:08.940716 11460 tablet_service.cc:1467] Processing CreateTablet for tablet be79bbe3be8a44c7b77333e72bc047e0 (DEFAULT_TABLE table=test-table [id=d5ebd541843340ea98d19abc791813c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 23:18:08.941082 11460 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be79bbe3be8a44c7b77333e72bc047e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:18:08.943373 11540 tablet_bootstrap.cc:492] T be79bbe3be8a44c7b77333e72bc047e0 P 6c447d4929ea488e81b6897b1d959580: Bootstrap starting.
I20260923 23:18:08.944082 11540 tablet_bootstrap.cc:654] T be79bbe3be8a44c7b77333e72bc047e0 P 6c447d4929ea488e81b6897b1d959580: Neither blocks nor log segments found. Creating new log.
I20260923 23:18:08.944084 11326 tablet_service.cc:1467] Processing CreateTablet for tablet be79bbe3be8a44c7b77333e72bc047e0 (DEFAULT_TABLE table=test-table [id=d5ebd541843340ea98d19abc791813c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 23:18:08.944144 11192 tablet_service.cc:1467] Processing CreateTablet for tablet be79bbe3be8a44c7b77333e72bc047e0 (DEFAULT_TABLE table=test-table [id=d5ebd541843340ea98d19abc791813c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 23:18:08.944409 11326 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be79bbe3be8a44c7b77333e72bc047e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:18:08.944495 11192 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be79bbe3be8a44c7b77333e72bc047e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:18:08.944620 11540 log.cc:824] T be79bbe3be8a44c7b77333e72bc047e0 P 6c447d4929ea488e81b6897b1d959580: Log is configured to *not* fsync() on all Append() calls
I20260923 23:18:08.945324 11540 tablet_bootstrap.cc:492] T be79bbe3be8a44c7b77333e72bc047e0 P 6c447d4929ea488e81b6897b1d959580: No bootstrap required, opened a new log
I20260923 23:18:08.945398 11540 ts_tablet_manager.cc:1397] T be79bbe3be8a44c7b77333e72bc047e0 P 6c447d4929ea488e81b6897b1d959580: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 23:18:08.946784 11542 tablet_bootstrap.cc:492] T be79bbe3be8a44c7b77333e72bc047e0 P 3e258479bc65400c995e200ec210d27b: Bootstrap starting.
I20260923 23:18:08.947148 11540 raft_consensus.cc:348] T be79bbe3be8a44c7b77333e72bc047e0 P 6c447d4929ea488e81b6897b1d959580 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e258479bc65400c995e200ec210d27b" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 36153 } } peers { permanent_uuid: "6c447d4929ea488e81b6897b1d959580" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 42475 } } peers { permanent_uuid: "0deb4a30e6bc41c39bd9c6cab1925f80" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 43587 } }
I20260923 23:18:08.947317 11540 raft_consensus.cc:374] T be79bbe3be8a44c7b77333e72bc047e0 P 6c447d4929ea488e81b6897b1d959580 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:18:08.947355 11540 raft_consensus.cc:729] T be79bbe3be8a44c7b77333e72bc047e0 P 6c447d4929ea488e81b6897b1d959580 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c447d4929ea488e81b6897b1d959580, State: Initialized, Role: FOLLOWER
I20260923 23:18:08.947419 11543 tablet_bootstrap.cc:492] T be79bbe3be8a44c7b77333e72bc047e0 P 0deb4a30e6bc41c39bd9c6cab1925f80: Bootstrap starting.
I20260923 23:18:08.947470 11540 consensus_queue.cc:260] T be79bbe3be8a44c7b77333e72bc047e0 P 6c447d4929ea488e81b6897b1d959580 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e258479bc65400c995e200ec210d27b" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 36153 } } peers { permanent_uuid: "6c447d4929ea488e81b6897b1d959580" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 42475 } } peers { permanent_uuid: "0deb4a30e6bc41c39bd9c6cab1925f80" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 43587 } }
I20260923 23:18:08.947711 11540 ts_tablet_manager.cc:1428] T be79bbe3be8a44c7b77333e72bc047e0 P 6c447d4929ea488e81b6897b1d959580: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260923 23:18:08.947664 11542 tablet_bootstrap.cc:654] T be79bbe3be8a44c7b77333e72bc047e0 P 3e258479bc65400c995e200ec210d27b: Neither blocks nor log segments found. Creating new log.
I20260923 23:18:08.947827 11526 heartbeater.cc:503] Master 127.24.227.254:34395 was elected leader, sending a full tablet report...
I20260923 23:18:08.948091 11543 tablet_bootstrap.cc:654] T be79bbe3be8a44c7b77333e72bc047e0 P 0deb4a30e6bc41c39bd9c6cab1925f80: Neither blocks nor log segments found. Creating new log.
I20260923 23:18:08.948308 11542 log.cc:824] T be79bbe3be8a44c7b77333e72bc047e0 P 3e258479bc65400c995e200ec210d27b: Log is configured to *not* fsync() on all Append() calls
I20260923 23:18:08.948621 11543 log.cc:824] T be79bbe3be8a44c7b77333e72bc047e0 P 0deb4a30e6bc41c39bd9c6cab1925f80: Log is configured to *not* fsync() on all Append() calls
I20260923 23:18:08.949071 11542 tablet_bootstrap.cc:492] T be79bbe3be8a44c7b77333e72bc047e0 P 3e258479bc65400c995e200ec210d27b: No bootstrap required, opened a new log
I20260923 23:18:08.949133 11542 ts_tablet_manager.cc:1397] T be79bbe3be8a44c7b77333e72bc047e0 P 3e258479bc65400c995e200ec210d27b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 23:18:08.949270 11543 tablet_bootstrap.cc:492] T be79bbe3be8a44c7b77333e72bc047e0 P 0deb4a30e6bc41c39bd9c6cab1925f80: No bootstrap required, opened a new log
I20260923 23:18:08.949324 11543 ts_tablet_manager.cc:1397] T be79bbe3be8a44c7b77333e72bc047e0 P 0deb4a30e6bc41c39bd9c6cab1925f80: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 23:18:08.951032 11542 raft_consensus.cc:348] T be79bbe3be8a44c7b77333e72bc047e0 P 3e258479bc65400c995e200ec210d27b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e258479bc65400c995e200ec210d27b" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 36153 } } peers { permanent_uuid: "6c447d4929ea488e81b6897b1d959580" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 42475 } } peers { permanent_uuid: "0deb4a30e6bc41c39bd9c6cab1925f80" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 43587 } }
I20260923 23:18:08.951217 11542 raft_consensus.cc:374] T be79bbe3be8a44c7b77333e72bc047e0 P 3e258479bc65400c995e200ec210d27b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:18:08.951272 11542 raft_consensus.cc:729] T be79bbe3be8a44c7b77333e72bc047e0 P 3e258479bc65400c995e200ec210d27b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e258479bc65400c995e200ec210d27b, State: Initialized, Role: FOLLOWER
I20260923 23:18:08.951224 11543 raft_consensus.cc:348] T be79bbe3be8a44c7b77333e72bc047e0 P 0deb4a30e6bc41c39bd9c6cab1925f80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e258479bc65400c995e200ec210d27b" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 36153 } } peers { permanent_uuid: "6c447d4929ea488e81b6897b1d959580" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 42475 } } peers { permanent_uuid: "0deb4a30e6bc41c39bd9c6cab1925f80" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 43587 } }
I20260923 23:18:08.951378 11543 raft_consensus.cc:374] T be79bbe3be8a44c7b77333e72bc047e0 P 0deb4a30e6bc41c39bd9c6cab1925f80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:18:08.951422 11543 raft_consensus.cc:729] T be79bbe3be8a44c7b77333e72bc047e0 P 0deb4a30e6bc41c39bd9c6cab1925f80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0deb4a30e6bc41c39bd9c6cab1925f80, State: Initialized, Role: FOLLOWER
I20260923 23:18:08.951385 11542 consensus_queue.cc:260] T be79bbe3be8a44c7b77333e72bc047e0 P 3e258479bc65400c995e200ec210d27b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e258479bc65400c995e200ec210d27b" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 36153 } } peers { permanent_uuid: "6c447d4929ea488e81b6897b1d959580" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 42475 } } peers { permanent_uuid: "0deb4a30e6bc41c39bd9c6cab1925f80" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 43587 } }
I20260923 23:18:08.951529 11543 consensus_queue.cc:260] T be79bbe3be8a44c7b77333e72bc047e0 P 0deb4a30e6bc41c39bd9c6cab1925f80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e258479bc65400c995e200ec210d27b" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 36153 } } peers { permanent_uuid: "6c447d4929ea488e81b6897b1d959580" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 42475 } } peers { permanent_uuid: "0deb4a30e6bc41c39bd9c6cab1925f80" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 43587 } }
I20260923 23:18:08.951648 11542 ts_tablet_manager.cc:1428] T be79bbe3be8a44c7b77333e72bc047e0 P 3e258479bc65400c995e200ec210d27b: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260923 23:18:08.951691 11392 heartbeater.cc:503] Master 127.24.227.254:34395 was elected leader, sending a full tablet report...
I20260923 23:18:08.952093 11543 ts_tablet_manager.cc:1428] T be79bbe3be8a44c7b77333e72bc047e0 P 0deb4a30e6bc41c39bd9c6cab1925f80: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 23:18:08.952154 11258 heartbeater.cc:503] Master 127.24.227.254:34395 was elected leader, sending a full tablet report...
W20260923 23:18:08.997942 11393 tablet.cc:2369] T be79bbe3be8a44c7b77333e72bc047e0 P 3e258479bc65400c995e200ec210d27b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 23:18:09.037889 11547 raft_consensus.cc:482] T be79bbe3be8a44c7b77333e72bc047e0 P 3e258479bc65400c995e200ec210d27b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 23:18:09.037989 11547 raft_consensus.cc:504] T be79bbe3be8a44c7b77333e72bc047e0 P 3e258479bc65400c995e200ec210d27b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e258479bc65400c995e200ec210d27b" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 36153 } } peers { permanent_uuid: "6c447d4929ea488e81b6897b1d959580" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 42475 } } peers { permanent_uuid: "0deb4a30e6bc41c39bd9c6cab1925f80" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 43587 } }
I20260923 23:18:09.038404 11547 leader_election.cc:288] T be79bbe3be8a44c7b77333e72bc047e0 P 3e258479bc65400c995e200ec210d27b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c447d4929ea488e81b6897b1d959580 (127.24.227.195:42475), 0deb4a30e6bc41c39bd9c6cab1925f80 (127.24.227.193:43587)
I20260923 23:18:09.041301 11394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:09.038590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.195:42475 (local address 127.24.227.194:47331)
0923 23:18:09.038644 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:09.038656 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 23:18:09.038706 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:18:09.038877 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:18:09.038878 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:18:09.038880 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:18:09.038949 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:09.038951 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:09.039712 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:09.039713 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:09.040470 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:09.040472 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:09.041040 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:09.041041 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:09.041137 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:09.041141 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 23:18:09.041195 (+ 54us) client_negotiation.cc:240] Negotiation successful
0923 23:18:09.041251 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260923 23:18:09.041357 11536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:09.038692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.194:47331 (local address 127.24.227.195:42475)
0923 23:18:09.038731 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:09.038733 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:09.038740 (+ 7us) server_negotiation.cc:407] Connection header received
0923 23:18:09.038767 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:09.038767 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:09.038776 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:09.038843 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:18:09.039062 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:09.039633 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:09.040557 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:09.040959 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:09.041013 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:09.041020 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:09.041232 (+ 212us) server_negotiation.cc:299] Negotiation successful
0923 23:18:09.041268 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260923 23:18:09.041505 11539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:09.038663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.194:44771 (local address 127.24.227.193:43587)
0923 23:18:09.038726 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:09.038728 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:09.038886 (+ 158us) server_negotiation.cc:407] Connection header received
0923 23:18:09.038915 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:09.038916 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:09.038927 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:09.039003 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:18:09.039202 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:09.039709 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:09.040674 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:09.041104 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:09.041179 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:09.041188 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:09.041377 (+ 189us) server_negotiation.cc:299] Negotiation successful
0923 23:18:09.041426 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260923 23:18:09.041667 11549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:09.038591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.193:43587 (local address 127.24.227.194:44771)
0923 23:18:09.038804 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:09.038813 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 23:18:09.038864 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:18:09.039037 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:18:09.039038 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:18:09.039040 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:18:09.039132 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:09.039134 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:09.039789 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:09.039790 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:09.040588 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:09.040590 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:09.041212 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:09.041213 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:09.041286 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:09.041290 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 23:18:09.041348 (+ 58us) client_negotiation.cc:240] Negotiation successful
0923 23:18:09.041531 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"mutex_wait_us":12,"thread_start_us":134,"threads_started":1}
I20260923 23:18:09.041654 11480 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be79bbe3be8a44c7b77333e72bc047e0" candidate_uuid: "3e258479bc65400c995e200ec210d27b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c447d4929ea488e81b6897b1d959580" is_pre_election: true
I20260923 23:18:09.041826 11480 raft_consensus.cc:2454] T be79bbe3be8a44c7b77333e72bc047e0 P 6c447d4929ea488e81b6897b1d959580 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e258479bc65400c995e200ec210d27b in term 0.
I20260923 23:18:09.042052 11282 leader_election.cc:302] T be79bbe3be8a44c7b77333e72bc047e0 P 3e258479bc65400c995e200ec210d27b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e258479bc65400c995e200ec210d27b, 6c447d4929ea488e81b6897b1d959580; no voters:
I20260923 23:18:09.042045 11212 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be79bbe3be8a44c7b77333e72bc047e0" candidate_uuid: "3e258479bc65400c995e200ec210d27b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0deb4a30e6bc41c39bd9c6cab1925f80" is_pre_election: true
I20260923 23:18:09.042181 11212 raft_consensus.cc:2454] T be79bbe3be8a44c7b77333e72bc047e0 P 0deb4a30e6bc41c39bd9c6cab1925f80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e258479bc65400c995e200ec210d27b in term 0.
I20260923 23:18:09.042184 11547 raft_consensus.cc:2781] T be79bbe3be8a44c7b77333e72bc047e0 P 3e258479bc65400c995e200ec210d27b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 23:18:09.042234 11547 raft_consensus.cc:482] T be79bbe3be8a44c7b77333e72bc047e0 P 3e258479bc65400c995e200ec210d27b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 23:18:09.042271 11547 raft_consensus.cc:3037] T be79bbe3be8a44c7b77333e72bc047e0 P 3e258479bc65400c995e200ec210d27b [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:18:09.042963 11547 raft_consensus.cc:504] T be79bbe3be8a44c7b77333e72bc047e0 P 3e258479bc65400c995e200ec210d27b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e258479bc65400c995e200ec210d27b" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 36153 } } peers { permanent_uuid: "6c447d4929ea488e81b6897b1d959580" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 42475 } } peers { permanent_uuid: "0deb4a30e6bc41c39bd9c6cab1925f80" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 43587 } }
I20260923 23:18:09.043104 11547 leader_election.cc:288] T be79bbe3be8a44c7b77333e72bc047e0 P 3e258479bc65400c995e200ec210d27b [CANDIDATE]: Term 1 election: Requested vote from peers 6c447d4929ea488e81b6897b1d959580 (127.24.227.195:42475), 0deb4a30e6bc41c39bd9c6cab1925f80 (127.24.227.193:43587)
I20260923 23:18:09.043247 11480 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be79bbe3be8a44c7b77333e72bc047e0" candidate_uuid: "3e258479bc65400c995e200ec210d27b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c447d4929ea488e81b6897b1d959580"
I20260923 23:18:09.043275 11212 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be79bbe3be8a44c7b77333e72bc047e0" candidate_uuid: "3e258479bc65400c995e200ec210d27b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0deb4a30e6bc41c39bd9c6cab1925f80"
I20260923 23:18:09.043319 11480 raft_consensus.cc:3037] T be79bbe3be8a44c7b77333e72bc047e0 P 6c447d4929ea488e81b6897b1d959580 [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:18:09.043358 11212 raft_consensus.cc:3037] T be79bbe3be8a44c7b77333e72bc047e0 P 0deb4a30e6bc41c39bd9c6cab1925f80 [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:18:09.044080 11480 raft_consensus.cc:2454] T be79bbe3be8a44c7b77333e72bc047e0 P 6c447d4929ea488e81b6897b1d959580 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e258479bc65400c995e200ec210d27b in term 1.
I20260923 23:18:09.044258 11212 raft_consensus.cc:2454] T be79bbe3be8a44c7b77333e72bc047e0 P 0deb4a30e6bc41c39bd9c6cab1925f80 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e258479bc65400c995e200ec210d27b in term 1.
I20260923 23:18:09.044253 11282 leader_election.cc:302] T be79bbe3be8a44c7b77333e72bc047e0 P 3e258479bc65400c995e200ec210d27b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e258479bc65400c995e200ec210d27b, 6c447d4929ea488e81b6897b1d959580; no voters:
I20260923 23:18:09.044436 11547 raft_consensus.cc:2781] T be79bbe3be8a44c7b77333e72bc047e0 P 3e258479bc65400c995e200ec210d27b [term 1 FOLLOWER]: Leader election won for term 1
I20260923 23:18:09.044622 11547 raft_consensus.cc:686] T be79bbe3be8a44c7b77333e72bc047e0 P 3e258479bc65400c995e200ec210d27b [term 1 LEADER]: Becoming Leader. State: Replica: 3e258479bc65400c995e200ec210d27b, State: Running, Role: LEADER
I20260923 23:18:09.044736 11547 consensus_queue.cc:237] T be79bbe3be8a44c7b77333e72bc047e0 P 3e258479bc65400c995e200ec210d27b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e258479bc65400c995e200ec210d27b" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 36153 } } peers { permanent_uuid: "6c447d4929ea488e81b6897b1d959580" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 42475 } } peers { permanent_uuid: "0deb4a30e6bc41c39bd9c6cab1925f80" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 43587 } }
I20260923 23:18:09.045490 11066 catalog_manager.cc:5495] T be79bbe3be8a44c7b77333e72bc047e0 P 3e258479bc65400c995e200ec210d27b reported cstate change: term changed from 0 to 1, leader changed from <none> to 3e258479bc65400c995e200ec210d27b (127.24.227.194). New cstate: current_term: 1 leader_uuid: "3e258479bc65400c995e200ec210d27b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e258479bc65400c995e200ec210d27b" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 36153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c447d4929ea488e81b6897b1d959580" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 42475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0deb4a30e6bc41c39bd9c6cab1925f80" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 43587 } health_report { overall_health: UNKNOWN } } }
I20260923 23:18:09.054418 11538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:09.052049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55644 (local address 127.24.227.194:36153)
0923 23:18:09.052105 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:09.052107 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:09.052118 (+ 11us) server_negotiation.cc:407] Connection header received
0923 23:18:09.052226 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:09.052227 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:09.052243 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:09.052309 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 23:18:09.052455 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:09.052883 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:09.053599 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:09.053809 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:09.053893 (+ 84us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:09.054082 (+ 189us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 23:18:09.054177 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 23:18:09.054290 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:09.054339 (+ 49us) server_negotiation.cc:299] Negotiation successful
0923 23:18:09.054348 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260923 23:18:09.055897 11480 raft_consensus.cc:1261] T be79bbe3be8a44c7b77333e72bc047e0 P 6c447d4929ea488e81b6897b1d959580 [term 1 FOLLOWER]: Refusing update from remote peer 3e258479bc65400c995e200ec210d27b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 23:18:09.055898 11212 raft_consensus.cc:1261] T be79bbe3be8a44c7b77333e72bc047e0 P 0deb4a30e6bc41c39bd9c6cab1925f80 [term 1 FOLLOWER]: Refusing update from remote peer 3e258479bc65400c995e200ec210d27b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 23:18:09.056248 11550 consensus_queue.cc:1035] T be79bbe3be8a44c7b77333e72bc047e0 P 3e258479bc65400c995e200ec210d27b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0deb4a30e6bc41c39bd9c6cab1925f80" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 43587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 23:18:09.056371 11547 consensus_queue.cc:1035] T be79bbe3be8a44c7b77333e72bc047e0 P 3e258479bc65400c995e200ec210d27b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c447d4929ea488e81b6897b1d959580" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 42475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 23:18:09.059034 11555 mvcc.cc:205] Tried to move back new op lower bound from 7332681683170820096 to 7332681683127836672. Current Snapshot: MvccSnapshot[applied={T|T < 7332681683170820096}]
I20260923 23:18:09.060858 11066 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35332:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 23:18:09.060948 11066 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35332:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 23:18:09.061661 11066 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f65fe1a63de64b22ad7c872564d22e1c: Sending DeleteTablet for 3 replicas of tablet be79bbe3be8a44c7b77333e72bc047e0
I20260923 23:18:09.061949 11326 tablet_service.cc:1514] Processing DeleteTablet for tablet be79bbe3be8a44c7b77333e72bc047e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 23:18:09 UTC) from {username='kudu', principal='kudu/127.24.227.254@KRBTEST.COM'} at 127.0.0.1:55632
I20260923 23:18:09.061986 11460 tablet_service.cc:1514] Processing DeleteTablet for tablet be79bbe3be8a44c7b77333e72bc047e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 23:18:09 UTC) from {username='kudu', principal='kudu/127.24.227.254@KRBTEST.COM'} at 127.0.0.1:45336
I20260923 23:18:09.062034 11192 tablet_service.cc:1514] Processing DeleteTablet for tablet be79bbe3be8a44c7b77333e72bc047e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 23:18:09 UTC) from {username='kudu', principal='kudu/127.24.227.254@KRBTEST.COM'} at 127.0.0.1:53284
I20260923 23:18:09.062199 11564 tablet_replica.cc:330] T be79bbe3be8a44c7b77333e72bc047e0 P 6c447d4929ea488e81b6897b1d959580: stopping tablet replica
I20260923 23:18:09.062315 11564 raft_consensus.cc:2229] T be79bbe3be8a44c7b77333e72bc047e0 P 6c447d4929ea488e81b6897b1d959580 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 23:18:09.062409 11564 raft_consensus.cc:2258] T be79bbe3be8a44c7b77333e72bc047e0 P 6c447d4929ea488e81b6897b1d959580 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 23:18:09.062399 11563 tablet_replica.cc:330] T be79bbe3be8a44c7b77333e72bc047e0 P 3e258479bc65400c995e200ec210d27b: stopping tablet replica
I20260923 23:18:09.062489 11563 raft_consensus.cc:2229] T be79bbe3be8a44c7b77333e72bc047e0 P 3e258479bc65400c995e200ec210d27b [term 1 LEADER]: Raft consensus shutting down.
I20260923 23:18:09.062495 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 11129
I20260923 23:18:09.062578 11563 raft_consensus.cc:2258] T be79bbe3be8a44c7b77333e72bc047e0 P 3e258479bc65400c995e200ec210d27b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 23:18:09.062765 11564 ts_tablet_manager.cc:1905] T be79bbe3be8a44c7b77333e72bc047e0 P 6c447d4929ea488e81b6897b1d959580: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 23:18:09.064193 11563 ts_tablet_manager.cc:1905] T be79bbe3be8a44c7b77333e72bc047e0 P 3e258479bc65400c995e200ec210d27b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 23:18:09.064463 11564 ts_tablet_manager.cc:1918] T be79bbe3be8a44c7b77333e72bc047e0 P 6c447d4929ea488e81b6897b1d959580: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 23:18:09.064522 11564 log.cc:1196] T be79bbe3be8a44c7b77333e72bc047e0 P 6c447d4929ea488e81b6897b1d959580: Deleting WAL directory at /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790205259943518-25487-0/minicluster-data/ts-2/wal/wals/be79bbe3be8a44c7b77333e72bc047e0
I20260923 23:18:09.064803 11564 ts_tablet_manager.cc:1939] T be79bbe3be8a44c7b77333e72bc047e0 P 6c447d4929ea488e81b6897b1d959580: Deleting consensus metadata
I20260923 23:18:09.065611 11054 catalog_manager.cc:4841] TS 6c447d4929ea488e81b6897b1d959580 (127.24.227.195:42475): tablet be79bbe3be8a44c7b77333e72bc047e0 (table test-table [id=d5ebd541843340ea98d19abc791813c5]) successfully deleted
I20260923 23:18:09.066162 11563 ts_tablet_manager.cc:1918] T be79bbe3be8a44c7b77333e72bc047e0 P 3e258479bc65400c995e200ec210d27b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 23:18:09.066233 11563 log.cc:1196] T be79bbe3be8a44c7b77333e72bc047e0 P 3e258479bc65400c995e200ec210d27b: Deleting WAL directory at /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790205259943518-25487-0/minicluster-data/ts-1/wal/wals/be79bbe3be8a44c7b77333e72bc047e0
I20260923 23:18:09.066510 11563 ts_tablet_manager.cc:1939] T be79bbe3be8a44c7b77333e72bc047e0 P 3e258479bc65400c995e200ec210d27b: Deleting consensus metadata
I20260923 23:18:09.066881 11053 catalog_manager.cc:4841] TS 3e258479bc65400c995e200ec210d27b (127.24.227.194:36153): tablet be79bbe3be8a44c7b77333e72bc047e0 (table test-table [id=d5ebd541843340ea98d19abc791813c5]) successfully deleted
W20260923 23:18:09.070230 11053 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.227.193:43587 (error 108)
I20260923 23:18:09.070676 11534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:09.070553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.193:43587 (local address 127.0.0.1:53288)
0923 23:18:09.070608 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:09.070647 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.193:43587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260923 23:18:09.070765 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 11263
W20260923 23:18:09.070792 11053 catalog_manager.cc:4584] TS 0deb4a30e6bc41c39bd9c6cab1925f80 (127.24.227.193:43587): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet be79bbe3be8a44c7b77333e72bc047e0: Network error: Client connection negotiation failed: client connection to 127.24.227.193:43587: connect: Connection refused (error 111)
I20260923 23:18:09.075074 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 11397
I20260923 23:18:09.079401 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 11035
2026-09-23T23:18:09Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (1908 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-09-23T23:18:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T23:18:09Z Disabled control of system clock
I20260923 23:18:09.105749 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790205259943518-25487-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790205259943518-25487-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790205259943518-25487-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.254:42709
--webserver_interface=127.24.227.254
--webserver_port=0
--builtin_ntp_servers=127.24.227.212:38957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.227.254:42709
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260923 23:18:09.184052 11570 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:18:09.184250 11570 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:18:09.184270 11570 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:18:09.185333 11570 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 23:18:09.185365 11570 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:18:09.185379 11570 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:18:09.185391 11570 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 23:18:09.185403 11570 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 23:18:09.186650 11570 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:38957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790205259943518-25487-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790205259943518-25487-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.227.254:42709
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.254:42709
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.227.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11570
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790205259943518-25487-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:18:09.186852 11570 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:18:09.187072 11570 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:18:09.191514 11578 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:18:09.191514 11576 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:18:09.191555 11575 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:18:09.191738 11570 server_base.cc:1034] running on GCE node
I20260923 23:18:09.192091 11570 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:18:09.192359 11570 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:18:09.193522 11570 hybrid_clock.cc:648] HybridClock initialized: now 1790205489193501 us; error 33 us; skew 500 ppm
I20260923 23:18:09.194729 11570 webserver.cc:460] Webserver started at http://127.24.227.254:38755/ using document root <none> and password file <none>
I20260923 23:18:09.194970 11570 fs_manager.cc:359] Metadata directory not provided
I20260923 23:18:09.195046 11570 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:18:09.195209 11570 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:18:09.196274 11570 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790205259943518-25487-0/minicluster-data/master-0/data/instance:
uuid: "21e710c992484e0eab85f4a3a0a9789d"
format_stamp: "Formatted at 2026-09-23 23:18:09 on dist-test-slave-754n"
server_key: "e7a406a9f228c6bff4b21ac0744ed7a3"
server_key_iv: "56037d8253b52ccccb082a779c18ea55"
server_key_version: "encryptionkey@0"
I20260923 23:18:09.196619 11570 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790205259943518-25487-0/minicluster-data/master-0/wal/instance:
uuid: "21e710c992484e0eab85f4a3a0a9789d"
format_stamp: "Formatted at 2026-09-23 23:18:09 on dist-test-slave-754n"
server_key: "e7a406a9f228c6bff4b21ac0744ed7a3"
server_key_iv: "56037d8253b52ccccb082a779c18ea55"
server_key_version: "encryptionkey@0"
I20260923 23:18:09.197824 11570 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 23:18:09.198477 11584 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:18:09.198637 11570 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 23:18:09.198745 11570 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790205259943518-25487-0/minicluster-data/master-0/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790205259943518-25487-0/minicluster-data/master-0/wal
uuid: "21e710c992484e0eab85f4a3a0a9789d"
format_stamp: "Formatted at 2026-09-23 23:18:09 on dist-test-slave-754n"
server_key: "e7a406a9f228c6bff4b21ac0744ed7a3"
server_key_iv: "56037d8253b52ccccb082a779c18ea55"
server_key_version: "encryptionkey@0"
I20260923 23:18:09.198817 11570 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790205259943518-25487-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790205259943518-25487-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790205259943518-25487-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:18:09.212903 11570 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:18:09.213165 11570 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:18:09.213313 11570 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:18:09.217161 11570 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.254:42709
I20260923 23:18:09.217190 11636 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.254:42709 every 8 connection(s)
I20260923 23:18:09.217494 11570 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
I20260923 23:18:09.218464 11637 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:18:09.219491 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 11570
I20260923 23:18:09.219589 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790205259943518-25487-0/minicluster-data/master-0/wal/instance
I20260923 23:18:09.219823 25487 external_mini_cluster.cc:1428] Setting key cd8e2c83d802ec95de9830ea5e64fd89
I20260923 23:18:09.221045 11637 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21e710c992484e0eab85f4a3a0a9789d: Bootstrap starting.
I20260923 23:18:09.221685 11637 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 21e710c992484e0eab85f4a3a0a9789d: Neither blocks nor log segments found. Creating new log.
I20260923 23:18:09.221973 11637 log.cc:824] T 00000000000000000000000000000000 P 21e710c992484e0eab85f4a3a0a9789d: Log is configured to *not* fsync() on all Append() calls
I20260923 23:18:09.222733 11637 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21e710c992484e0eab85f4a3a0a9789d: No bootstrap required, opened a new log
I20260923 23:18:09.224992 11637 raft_consensus.cc:348] T 00000000000000000000000000000000 P 21e710c992484e0eab85f4a3a0a9789d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21e710c992484e0eab85f4a3a0a9789d" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 42709 } }
I20260923 23:18:09.225160 11637 raft_consensus.cc:374] T 00000000000000000000000000000000 P 21e710c992484e0eab85f4a3a0a9789d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:18:09.225255 11637 raft_consensus.cc:729] T 00000000000000000000000000000000 P 21e710c992484e0eab85f4a3a0a9789d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21e710c992484e0eab85f4a3a0a9789d, State: Initialized, Role: FOLLOWER
I20260923 23:18:09.225381 11640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:09.220330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38082 (local address 127.24.227.254:42709)
0923 23:18:09.220506 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:09.220512 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:09.220535 (+ 23us) server_negotiation.cc:407] Connection header received
0923 23:18:09.220917 (+ 382us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:09.220923 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:09.220977 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:09.221103 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:09.221328 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:09.222343 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:09.223468 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:09.223936 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:09.224033 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:09.224647 (+ 614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:18:09.224650 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:18:09.224650 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 23:18:09.224806 (+ 156us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:18:09.224835 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 23:18:09.224845 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:18:09.224891 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:09.224991 (+ 100us) server_negotiation.cc:299] Negotiation successful
0923 23:18:09.225074 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":83,"threads_started":1}
I20260923 23:18:09.225425 11637 consensus_queue.cc:260] T 00000000000000000000000000000000 P 21e710c992484e0eab85f4a3a0a9789d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21e710c992484e0eab85f4a3a0a9789d" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 42709 } }
I20260923 23:18:09.225498 11637 raft_consensus.cc:388] T 00000000000000000000000000000000 P 21e710c992484e0eab85f4a3a0a9789d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 23:18:09.225530 11637 raft_consensus.cc:482] T 00000000000000000000000000000000 P 21e710c992484e0eab85f4a3a0a9789d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 23:18:09.225579 11637 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 21e710c992484e0eab85f4a3a0a9789d [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:18:09.226214 11637 raft_consensus.cc:504] T 00000000000000000000000000000000 P 21e710c992484e0eab85f4a3a0a9789d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21e710c992484e0eab85f4a3a0a9789d" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 42709 } }
I20260923 23:18:09.226325 11637 leader_election.cc:302] T 00000000000000000000000000000000 P 21e710c992484e0eab85f4a3a0a9789d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 21e710c992484e0eab85f4a3a0a9789d; no voters:
I20260923 23:18:09.226470 11637 leader_election.cc:288] T 00000000000000000000000000000000 P 21e710c992484e0eab85f4a3a0a9789d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 23:18:09.226531 11642 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 21e710c992484e0eab85f4a3a0a9789d [term 1 FOLLOWER]: Leader election won for term 1
I20260923 23:18:09.226667 11642 raft_consensus.cc:686] T 00000000000000000000000000000000 P 21e710c992484e0eab85f4a3a0a9789d [term 1 LEADER]: Becoming Leader. State: Replica: 21e710c992484e0eab85f4a3a0a9789d, State: Running, Role: LEADER
I20260923 23:18:09.226720 11637 sys_catalog.cc:564] T 00000000000000000000000000000000 P 21e710c992484e0eab85f4a3a0a9789d [sys.catalog]: configured and running, proceeding with master startup.
I20260923 23:18:09.226750 11642 consensus_queue.cc:237] T 00000000000000000000000000000000 P 21e710c992484e0eab85f4a3a0a9789d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21e710c992484e0eab85f4a3a0a9789d" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 42709 } }
I20260923 23:18:09.227108 11644 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21e710c992484e0eab85f4a3a0a9789d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 21e710c992484e0eab85f4a3a0a9789d. Latest consensus state: current_term: 1 leader_uuid: "21e710c992484e0eab85f4a3a0a9789d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21e710c992484e0eab85f4a3a0a9789d" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 42709 } } }
I20260923 23:18:09.227210 11644 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21e710c992484e0eab85f4a3a0a9789d [sys.catalog]: This master's current role is: LEADER
I20260923 23:18:09.227437 11643 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21e710c992484e0eab85f4a3a0a9789d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "21e710c992484e0eab85f4a3a0a9789d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21e710c992484e0eab85f4a3a0a9789d" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 42709 } } }
I20260923 23:18:09.227541 11643 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21e710c992484e0eab85f4a3a0a9789d [sys.catalog]: This master's current role is: LEADER
W20260923 23:18:09.228415 11658 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 21e710c992484e0eab85f4a3a0a9789d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 23:18:09.228477 11658 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 23:18:09.228525 11650 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 23:18:09.228685 11650 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 23:18:09.229929 11650 catalog_manager.cc:1347] Generated new cluster ID: 3897030651a54408ac1db4aaaf5b2081
I20260923 23:18:09.229976 11650 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 23:18:09.243634 11650 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 23:18:09.244202 11650 catalog_manager.cc:1504] Loading token signing keys...
I20260923 23:18:09.250895 11650 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 21e710c992484e0eab85f4a3a0a9789d: Generated new TSK 0
I20260923 23:18:09.251076 11650 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 23:18:09.252311 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790205259943518-25487-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790205259943518-25487-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790205259943518-25487-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790205259943518-25487-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.227.193:0
--local_ip_for_outbound_sockets=127.24.227.193
--webserver_interface=127.24.227.193
--webserver_port=0
--tserver_master_addrs=127.24.227.254:42709
--builtin_ntp_servers=127.24.227.212:38957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260923 23:18:09.331051 11661 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:18:09.331274 11661 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:18:09.331310 11661 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:18:09.332448 11661 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:18:09.332523 11661 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:18:09.332564 11661 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.227.193
I20260923 23:18:09.334081 11661 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:38957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790205259943518-25487-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790205259943518-25487-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.227.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790205259943518-25487-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.227.193
--webserver_port=0
--tserver_master_addrs=127.24.227.254:42709
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11661
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.227.193
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790205259943518-25487-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:18:09.334332 11661 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:18:09.334610 11661 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:18:09.340160 11667 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:18:09.340219 11669 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:18:09.340169 11666 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:18:09.340586 11661 server_base.cc:1034] running on GCE node
I20260923 23:18:09.340801 11661 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:18:09.341061 11661 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:18:09.342226 11661 hybrid_clock.cc:648] HybridClock initialized: now 1790205489342215 us; error 26 us; skew 500 ppm
I20260923 23:18:09.343463 11661 webserver.cc:460] Webserver started at http://127.24.227.193:42643/ using document root <none> and password file <none>
I20260923 23:18:09.343708 11661 fs_manager.cc:359] Metadata directory not provided
I20260923 23:18:09.343779 11661 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:18:09.343954 11661 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:18:09.345044 11661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790205259943518-25487-0/minicluster-data/ts-0/data/instance:
uuid: "ad4d550a1abc4b94a458f42ba15d8112"
format_stamp: "Formatted at 2026-09-23 23:18:09 on dist-test-slave-754n"
server_key: "53dab2610ef891b3474702c836a86161"
server_key_iv: "407f25c1f8ba229bad63dbec1d2b25d4"
server_key_version: "encryptionkey@0"
I20260923 23:18:09.345402 11661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790205259943518-25487-0/minicluster-data/ts-0/wal/instance:
uuid: "ad4d550a1abc4b94a458f42ba15d8112"
format_stamp: "Formatted at 2026-09-23 23:18:09 on dist-test-slave-754n"
server_key: "53dab2610ef891b3474702c836a86161"
server_key_iv: "407f25c1f8ba229bad63dbec1d2b25d4"
server_key_version: "encryptionkey@0"
I20260923 23:18:09.346758 11661 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 23:18:09.347512 11675 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:18:09.347744 11661 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 23:18:09.347841 11661 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790205259943518-25487-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790205259943518-25487-0/minicluster-data/ts-0/wal
uuid: "ad4d550a1abc4b94a458f42ba15d8112"
format_stamp: "Formatted at 2026-09-23 23:18:09 on dist-test-slave-754n"
server_key: "53dab2610ef891b3474702c836a86161"
server_key_iv: "407f25c1f8ba229bad63dbec1d2b25d4"
server_key_version: "encryptionkey@0"
I20260923 23:18:09.347944 11661 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790205259943518-25487-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790205259943518-25487-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790205259943518-25487-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:18:09.376055 11661 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:18:09.376304 11661 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:18:09.376405 11661 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:18:09.376596 11661 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 23:18:09.376861 11661 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 23:18:09.376890 11661 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 23:18:09.376910 11661 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 23:18:09.376922 11661 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 23:18:09.382522 11661 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.193:44595
I20260923 23:18:09.382568 11788 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.193:44595 every 8 connection(s)
I20260923 23:18:09.382879 11661 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790205259943518-25487-0/minicluster-data/ts-0/data/info.pb
I20260923 23:18:09.387205 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 11661
I20260923 23:18:09.387336 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790205259943518-25487-0/minicluster-data/ts-0/wal/instance
I20260923 23:18:09.387557 25487 external_mini_cluster.cc:1428] Setting key 79f0984b24d2bb996d6d28e21c824b4b
I20260923 23:18:09.387943 11640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:09.383552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.193:37907 (local address 127.24.227.254:42709)
0923 23:18:09.383632 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:09.383634 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:09.383846 (+ 212us) server_negotiation.cc:407] Connection header received
0923 23:18:09.384399 (+ 553us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:09.384401 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:09.384411 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:09.384465 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:09.385094 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:09.385588 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:09.386506 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:09.386694 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:09.386745 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:09.387592 (+ 847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:18:09.387593 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:18:09.387594 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 23:18:09.387615 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:18:09.387632 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 23:18:09.387637 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:18:09.387710 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:09.387805 (+ 95us) server_negotiation.cc:299] Negotiation successful
0923 23:18:09.387831 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260923 23:18:09.388163 11791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:09.383518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:42709 (local address 127.24.227.193:37907)
0923 23:18:09.383727 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:09.383760 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0923 23:18:09.384304 (+ 544us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:18:09.384520 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:18:09.384523 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:18:09.384532 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:18:09.385030 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:09.385033 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:09.385664 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:09.385665 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:09.386438 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:09.386439 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:09.386765 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:09.386766 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:09.386869 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:09.387435 (+ 566us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 23:18:09.387435 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:18:09.387478 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 23:18:09.387507 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:18:09.387717 (+ 210us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:18:09.387725 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:18:09.387728 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 23:18:09.387803 (+ 75us) client_negotiation.cc:240] Negotiation successful
0923 23:18:09.387929 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":105,"threads_started":1}
I20260923 23:18:09.388674 11789 heartbeater.cc:347] Connected to a master server at 127.24.227.254:42709
I20260923 23:18:09.388746 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790205259943518-25487-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790205259943518-25487-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790205259943518-25487-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790205259943518-25487-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.227.194:0
--local_ip_for_outbound_sockets=127.24.227.194
--webserver_interface=127.24.227.194
--webserver_port=0
--tserver_master_addrs=127.24.227.254:42709
--builtin_ntp_servers=127.24.227.212:38957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260923 23:18:09.388790 11789 heartbeater.cc:464] Registering TS with master...
I20260923 23:18:09.388984 11789 heartbeater.cc:511] Master 127.24.227.254:42709 requested a full tablet report, sending...
I20260923 23:18:09.389503 11601 ts_manager.cc:194] Registered new tserver with Master: ad4d550a1abc4b94a458f42ba15d8112 (127.24.227.193:44595)
I20260923 23:18:09.390112 11601 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.227.193:37907
W20260923 23:18:09.467199 11792 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:18:09.467372 11792 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:18:09.467391 11792 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:18:09.468540 11792 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:18:09.468576 11792 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:18:09.468600 11792 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.227.194
I20260923 23:18:09.469903 11792 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:38957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790205259943518-25487-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790205259943518-25487-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.227.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790205259943518-25487-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.227.194
--webserver_port=0
--tserver_master_addrs=127.24.227.254:42709
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11792
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.227.194
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790205259943518-25487-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:18:09.470148 11792 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:18:09.470436 11792 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:18:09.475136 11798 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:18:09.475171 11800 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:18:09.475304 11792 server_base.cc:1034] running on GCE node
W20260923 23:18:09.475174 11797 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:18:09.475598 11792 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:18:09.475823 11792 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:18:09.476991 11792 hybrid_clock.cc:648] HybridClock initialized: now 1790205489476975 us; error 36 us; skew 500 ppm
I20260923 23:18:09.478142 11792 webserver.cc:460] Webserver started at http://127.24.227.194:33153/ using document root <none> and password file <none>
I20260923 23:18:09.478492 11792 fs_manager.cc:359] Metadata directory not provided
I20260923 23:18:09.478575 11792 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:18:09.478710 11792 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:18:09.479717 11792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790205259943518-25487-0/minicluster-data/ts-1/data/instance:
uuid: "914e3b58d0c643db820d993c9bee73d5"
format_stamp: "Formatted at 2026-09-23 23:18:09 on dist-test-slave-754n"
server_key: "c6ffc81a641fc171d49ae13feb33e1a6"
server_key_iv: "4e43309d8e29c7cf13ac09dfe571d413"
server_key_version: "encryptionkey@0"
I20260923 23:18:09.480068 11792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790205259943518-25487-0/minicluster-data/ts-1/wal/instance:
uuid: "914e3b58d0c643db820d993c9bee73d5"
format_stamp: "Formatted at 2026-09-23 23:18:09 on dist-test-slave-754n"
server_key: "c6ffc81a641fc171d49ae13feb33e1a6"
server_key_iv: "4e43309d8e29c7cf13ac09dfe571d413"
server_key_version: "encryptionkey@0"
I20260923 23:18:09.481382 11792 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 23:18:09.482745 11806 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:18:09.482924 11792 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 23:18:09.483021 11792 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790205259943518-25487-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790205259943518-25487-0/minicluster-data/ts-1/wal
uuid: "914e3b58d0c643db820d993c9bee73d5"
format_stamp: "Formatted at 2026-09-23 23:18:09 on dist-test-slave-754n"
server_key: "c6ffc81a641fc171d49ae13feb33e1a6"
server_key_iv: "4e43309d8e29c7cf13ac09dfe571d413"
server_key_version: "encryptionkey@0"
I20260923 23:18:09.483091 11792 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790205259943518-25487-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790205259943518-25487-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790205259943518-25487-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:18:09.502893 11792 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:18:09.503105 11792 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:18:09.503192 11792 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:18:09.503341 11792 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 23:18:09.503561 11792 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 23:18:09.503588 11792 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 23:18:09.503607 11792 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 23:18:09.503619 11792 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 23:18:09.508893 11792 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.194:37211
I20260923 23:18:09.508960 11919 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.194:37211 every 8 connection(s)
I20260923 23:18:09.509261 11792 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790205259943518-25487-0/minicluster-data/ts-1/data/info.pb
I20260923 23:18:09.513497 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 11792
I20260923 23:18:09.513626 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790205259943518-25487-0/minicluster-data/ts-1/wal/instance
I20260923 23:18:09.513897 25487 external_mini_cluster.cc:1428] Setting key ecd5e2304e35eb5bfeb0cb15c119cb8c
I20260923 23:18:09.514408 11640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:09.510009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.194:34161 (local address 127.24.227.254:42709)
0923 23:18:09.510073 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:09.510075 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:09.510305 (+ 230us) server_negotiation.cc:407] Connection header received
0923 23:18:09.510734 (+ 429us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:09.510735 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:09.510744 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:09.510799 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:09.511393 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:09.511918 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:09.512874 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:09.513057 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:09.513110 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:09.514109 (+ 999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:18:09.514111 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:18:09.514111 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 23:18:09.514117 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:18:09.514136 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 23:18:09.514141 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:18:09.514197 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:09.514306 (+ 109us) server_negotiation.cc:299] Negotiation successful
0923 23:18:09.514331 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260923 23:18:09.514603 11922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:09.509979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:42709 (local address 127.24.227.194:34161)
0923 23:18:09.510168 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:09.510219 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0923 23:18:09.510652 (+ 433us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:18:09.510851 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:18:09.510854 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:18:09.510861 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:18:09.511328 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:09.511332 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:09.512013 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:09.512013 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:09.512796 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:09.512798 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:09.513133 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:09.513133 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:09.513245 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:09.513892 (+ 647us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 23:18:09.513892 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:18:09.513929 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 23:18:09.513960 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:18:09.514217 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:18:09.514222 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:18:09.514225 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 23:18:09.514291 (+ 66us) client_negotiation.cc:240] Negotiation successful
0923 23:18:09.514388 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":82,"threads_started":1}
I20260923 23:18:09.514986 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790205259943518-25487-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790205259943518-25487-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790205259943518-25487-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790205259943518-25487-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.227.195:0
--local_ip_for_outbound_sockets=127.24.227.195
--webserver_interface=127.24.227.195
--webserver_port=0
--tserver_master_addrs=127.24.227.254:42709
--builtin_ntp_servers=127.24.227.212:38957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260923 23:18:09.515082 11920 heartbeater.cc:347] Connected to a master server at 127.24.227.254:42709
I20260923 23:18:09.515190 11920 heartbeater.cc:464] Registering TS with master...
I20260923 23:18:09.515372 11920 heartbeater.cc:511] Master 127.24.227.254:42709 requested a full tablet report, sending...
I20260923 23:18:09.515723 11601 ts_manager.cc:194] Registered new tserver with Master: 914e3b58d0c643db820d993c9bee73d5 (127.24.227.194:37211)
I20260923 23:18:09.516258 11601 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.227.194:34161
W20260923 23:18:09.592479 11923 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:18:09.592687 11923 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:18:09.592722 11923 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:18:09.593783 11923 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:18:09.593849 11923 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:18:09.593892 11923 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.227.195
I20260923 23:18:09.595176 11923 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:38957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790205259943518-25487-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790205259943518-25487-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.227.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790205259943518-25487-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.227.195
--webserver_port=0
--tserver_master_addrs=127.24.227.254:42709
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11923
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.227.195
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790205259943518-25487-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:18:09.595407 11923 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:18:09.595655 11923 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:18:09.600211 11931 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:18:09.600203 11929 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:18:09.600252 11928 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:18:09.600486 11923 server_base.cc:1034] running on GCE node
I20260923 23:18:09.600680 11923 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:18:09.600970 11923 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:18:09.602129 11923 hybrid_clock.cc:648] HybridClock initialized: now 1790205489602120 us; error 42 us; skew 500 ppm
I20260923 23:18:09.606019 11923 webserver.cc:460] Webserver started at http://127.24.227.195:37411/ using document root <none> and password file <none>
I20260923 23:18:09.606274 11923 fs_manager.cc:359] Metadata directory not provided
I20260923 23:18:09.606318 11923 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:18:09.606464 11923 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:18:09.607409 11923 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790205259943518-25487-0/minicluster-data/ts-2/data/instance:
uuid: "b4ca6c55eb5e498bb975081d9e82e864"
format_stamp: "Formatted at 2026-09-23 23:18:09 on dist-test-slave-754n"
server_key: "efc0e06d1bdb1f5ca2dd6aa21e09533f"
server_key_iv: "bc8e101a93f846ac288410b24e2c33d1"
server_key_version: "encryptionkey@0"
I20260923 23:18:09.607731 11923 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790205259943518-25487-0/minicluster-data/ts-2/wal/instance:
uuid: "b4ca6c55eb5e498bb975081d9e82e864"
format_stamp: "Formatted at 2026-09-23 23:18:09 on dist-test-slave-754n"
server_key: "efc0e06d1bdb1f5ca2dd6aa21e09533f"
server_key_iv: "bc8e101a93f846ac288410b24e2c33d1"
server_key_version: "encryptionkey@0"
I20260923 23:18:09.608968 11923 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 23:18:09.609637 11937 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:18:09.609786 11923 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 23:18:09.609877 11923 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790205259943518-25487-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790205259943518-25487-0/minicluster-data/ts-2/wal
uuid: "b4ca6c55eb5e498bb975081d9e82e864"
format_stamp: "Formatted at 2026-09-23 23:18:09 on dist-test-slave-754n"
server_key: "efc0e06d1bdb1f5ca2dd6aa21e09533f"
server_key_iv: "bc8e101a93f846ac288410b24e2c33d1"
server_key_version: "encryptionkey@0"
I20260923 23:18:09.609967 11923 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790205259943518-25487-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790205259943518-25487-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790205259943518-25487-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:18:09.618965 11923 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:18:09.619278 11923 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:18:09.619436 11923 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:18:09.619642 11923 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 23:18:09.620007 11923 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 23:18:09.620052 11923 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 23:18:09.620079 11923 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 23:18:09.620095 11923 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 23:18:09.626190 11923 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.195:44423
I20260923 23:18:09.626231 12050 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.195:44423 every 8 connection(s)
I20260923 23:18:09.626591 11923 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790205259943518-25487-0/minicluster-data/ts-2/data/info.pb
I20260923 23:18:09.629446 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 11923
I20260923 23:18:09.629524 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790205259943518-25487-0/minicluster-data/ts-2/wal/instance
I20260923 23:18:09.629731 25487 external_mini_cluster.cc:1428] Setting key c5eaca4731f1357688f7408834237915
I20260923 23:18:09.631767 11640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:09.627291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.195:49095 (local address 127.24.227.254:42709)
0923 23:18:09.627362 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:09.627364 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:09.627524 (+ 160us) server_negotiation.cc:407] Connection header received
0923 23:18:09.628111 (+ 587us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:09.628112 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:09.628121 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:09.628170 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:09.628750 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:09.629226 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:09.630509 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:09.630750 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:09.630803 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:09.631580 (+ 777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:18:09.631581 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:18:09.631581 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 23:18:09.631584 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:18:09.631594 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 23:18:09.631597 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:18:09.631633 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:09.631695 (+ 62us) server_negotiation.cc:299] Negotiation successful
0923 23:18:09.631713 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260923 23:18:09.632035 12053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:09.627267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:42709 (local address 127.24.227.195:49095)
0923 23:18:09.627418 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:09.627447 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0923 23:18:09.628036 (+ 589us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:18:09.628220 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:18:09.628223 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:18:09.628230 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:18:09.628684 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:09.628688 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:09.629314 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:09.629315 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:09.630414 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:09.630416 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:09.630818 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:09.630818 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:09.630928 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:09.631454 (+ 526us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 23:18:09.631455 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:18:09.631488 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 23:18:09.631508 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:18:09.631635 (+ 127us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:18:09.631637 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:18:09.631639 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 23:18:09.631691 (+ 52us) client_negotiation.cc:240] Negotiation successful
0923 23:18:09.631759 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":76,"threads_started":1}
I20260923 23:18:09.632498 12051 heartbeater.cc:347] Connected to a master server at 127.24.227.254:42709
I20260923 23:18:09.632597 12051 heartbeater.cc:464] Registering TS with master...
I20260923 23:18:09.632787 12051 heartbeater.cc:511] Master 127.24.227.254:42709 requested a full tablet report, sending...
I20260923 23:18:09.633091 11601 ts_manager.cc:194] Registered new tserver with Master: b4ca6c55eb5e498bb975081d9e82e864 (127.24.227.195:44423)
I20260923 23:18:09.633461 11601 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.227.195:49095
I20260923 23:18:09.641307 25487 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 23:18:09.645488 11640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:09.642563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38084 (local address 127.24.227.254:42709)
0923 23:18:09.642638 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:09.642640 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:09.642737 (+ 97us) server_negotiation.cc:407] Connection header received
0923 23:18:09.642828 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:09.642829 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:09.642838 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:09.642891 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:09.643049 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:09.643504 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:09.644321 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:09.644520 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:09.644602 (+ 82us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:09.645173 (+ 571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:18:09.645174 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:18:09.645175 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 23:18:09.645179 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:18:09.645193 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 23:18:09.645197 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:18:09.645310 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:09.645401 (+ 91us) server_negotiation.cc:299] Negotiation successful
0923 23:18:09.645411 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260923 23:18:09.646904 11601 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38084:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 23:18:09.648497 11601 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 23:18:09.654759 12062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:09.650676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56802 (local address 127.24.227.194:37211)
0923 23:18:09.650875 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:09.650880 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:09.651067 (+ 187us) server_negotiation.cc:407] Connection header received
0923 23:18:09.651149 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:09.651151 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:09.651171 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:09.651258 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:18:09.651478 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:09.652568 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:09.653663 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:09.654386 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:09.654467 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:09.654554 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:09.654634 (+ 80us) server_negotiation.cc:299] Negotiation successful
0923 23:18:09.654670 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":90,"threads_started":1}
I20260923 23:18:09.654758 12060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:09.650507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.194:37211 (local address 127.0.0.1:56802)
0923 23:18:09.650975 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:09.650985 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 23:18:09.651069 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:18:09.651330 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:18:09.651331 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:18:09.651334 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:18:09.651381 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:09.651382 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:09.652634 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:09.652635 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:09.653578 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:09.653580 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:09.654470 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:09.654470 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:09.654520 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:09.654530 (+ 10us) client_negotiation.cc:769] Sending connection context
0923 23:18:09.654620 (+ 90us) client_negotiation.cc:240] Negotiation successful
0923 23:18:09.654668 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":453,"mutex_wait_us":2,"thread_start_us":73,"threads_started":1}
I20260923 23:18:09.654943 12063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:09.650507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.195:44423 (local address 127.0.0.1:58486)
0923 23:18:09.650833 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:09.650869 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0923 23:18:09.650937 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:18:09.651237 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:18:09.651240 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:18:09.651243 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:18:09.651296 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:09.651297 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:09.652542 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:09.652543 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:09.653788 (+ 1245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:09.653790 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:09.654614 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:09.654615 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:09.654665 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:09.654668 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 23:18:09.654719 (+ 51us) client_negotiation.cc:240] Negotiation successful
0923 23:18:09.654749 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"mutex_wait_us":42,"thread_start_us":55,"threads_started":1}
I20260923 23:18:09.655104 12061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:09.650582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58486 (local address 127.24.227.195:44423)
0923 23:18:09.650839 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:09.650844 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:09.651013 (+ 169us) server_negotiation.cc:407] Connection header received
0923 23:18:09.651055 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:09.651057 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:09.651069 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:09.651170 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:18:09.651376 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:09.652470 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:09.653871 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:09.654524 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:09.654600 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:09.654681 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:09.654794 (+ 113us) server_negotiation.cc:299] Negotiation successful
0923 23:18:09.655011 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":222,"thread_start_us":104,"threads_started":1}
I20260923 23:18:09.655534 11854 tablet_service.cc:1467] Processing CreateTablet for tablet 7063a475533148c5b74a8f3602f0a7e8 (DEFAULT_TABLE table=test-table [id=7e000f6c74024f03b8d6519f016e629a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 23:18:09.655711 11985 tablet_service.cc:1467] Processing CreateTablet for tablet 7063a475533148c5b74a8f3602f0a7e8 (DEFAULT_TABLE table=test-table [id=7e000f6c74024f03b8d6519f016e629a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 23:18:09.655930 12059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:09.650696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.193:44595 (local address 127.0.0.1:60772)
0923 23:18:09.651147 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:09.651155 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 23:18:09.651197 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:18:09.651447 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:18:09.651448 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:18:09.651450 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:18:09.651482 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:09.651483 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:09.652442 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:09.652443 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:09.653796 (+ 1353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:09.653797 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:09.655638 (+ 1841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:09.655639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:09.655732 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:09.655736 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 23:18:09.655789 (+ 53us) client_negotiation.cc:240] Negotiation successful
0923 23:18:09.655828 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":441,"mutex_wait_us":117,"thread_start_us":77,"threads_started":1}
I20260923 23:18:09.655936 11854 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7063a475533148c5b74a8f3602f0a7e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:18:09.656005 11985 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7063a475533148c5b74a8f3602f0a7e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:18:09.656216 12064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:09.650799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60772 (local address 127.24.227.193:44595)
0923 23:18:09.650963 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:09.650967 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:09.651255 (+ 288us) server_negotiation.cc:407] Connection header received
0923 23:18:09.651298 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:09.651300 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:09.651315 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:09.651382 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:18:09.651535 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:09.652362 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:09.654726 (+ 2364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:09.655464 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:09.655525 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:09.655606 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:09.656093 (+ 487us) server_negotiation.cc:299] Negotiation successful
0923 23:18:09.656135 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":80,"threads_started":1}
I20260923 23:18:09.656739 11723 tablet_service.cc:1467] Processing CreateTablet for tablet 7063a475533148c5b74a8f3602f0a7e8 (DEFAULT_TABLE table=test-table [id=7e000f6c74024f03b8d6519f016e629a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 23:18:09.657047 11723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7063a475533148c5b74a8f3602f0a7e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:18:09.659207 12066 tablet_bootstrap.cc:492] T 7063a475533148c5b74a8f3602f0a7e8 P b4ca6c55eb5e498bb975081d9e82e864: Bootstrap starting.
I20260923 23:18:09.659832 12067 tablet_bootstrap.cc:492] T 7063a475533148c5b74a8f3602f0a7e8 P ad4d550a1abc4b94a458f42ba15d8112: Bootstrap starting.
I20260923 23:18:09.659991 12066 tablet_bootstrap.cc:654] T 7063a475533148c5b74a8f3602f0a7e8 P b4ca6c55eb5e498bb975081d9e82e864: Neither blocks nor log segments found. Creating new log.
I20260923 23:18:09.660296 12066 log.cc:824] T 7063a475533148c5b74a8f3602f0a7e8 P b4ca6c55eb5e498bb975081d9e82e864: Log is configured to *not* fsync() on all Append() calls
I20260923 23:18:09.660444 12067 tablet_bootstrap.cc:654] T 7063a475533148c5b74a8f3602f0a7e8 P ad4d550a1abc4b94a458f42ba15d8112: Neither blocks nor log segments found. Creating new log.
I20260923 23:18:09.660624 12065 tablet_bootstrap.cc:492] T 7063a475533148c5b74a8f3602f0a7e8 P 914e3b58d0c643db820d993c9bee73d5: Bootstrap starting.
I20260923 23:18:09.660770 12067 log.cc:824] T 7063a475533148c5b74a8f3602f0a7e8 P ad4d550a1abc4b94a458f42ba15d8112: Log is configured to *not* fsync() on all Append() calls
I20260923 23:18:09.661055 12066 tablet_bootstrap.cc:492] T 7063a475533148c5b74a8f3602f0a7e8 P b4ca6c55eb5e498bb975081d9e82e864: No bootstrap required, opened a new log
I20260923 23:18:09.661144 12066 ts_tablet_manager.cc:1397] T 7063a475533148c5b74a8f3602f0a7e8 P b4ca6c55eb5e498bb975081d9e82e864: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260923 23:18:09.661269 12065 tablet_bootstrap.cc:654] T 7063a475533148c5b74a8f3602f0a7e8 P 914e3b58d0c643db820d993c9bee73d5: Neither blocks nor log segments found. Creating new log.
I20260923 23:18:09.661504 12067 tablet_bootstrap.cc:492] T 7063a475533148c5b74a8f3602f0a7e8 P ad4d550a1abc4b94a458f42ba15d8112: No bootstrap required, opened a new log
I20260923 23:18:09.661576 12067 ts_tablet_manager.cc:1397] T 7063a475533148c5b74a8f3602f0a7e8 P ad4d550a1abc4b94a458f42ba15d8112: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 23:18:09.661585 12065 log.cc:824] T 7063a475533148c5b74a8f3602f0a7e8 P 914e3b58d0c643db820d993c9bee73d5: Log is configured to *not* fsync() on all Append() calls
I20260923 23:18:09.662569 12065 tablet_bootstrap.cc:492] T 7063a475533148c5b74a8f3602f0a7e8 P 914e3b58d0c643db820d993c9bee73d5: No bootstrap required, opened a new log
I20260923 23:18:09.662635 12065 ts_tablet_manager.cc:1397] T 7063a475533148c5b74a8f3602f0a7e8 P 914e3b58d0c643db820d993c9bee73d5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 23:18:09.663260 12066 raft_consensus.cc:348] T 7063a475533148c5b74a8f3602f0a7e8 P b4ca6c55eb5e498bb975081d9e82e864 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4ca6c55eb5e498bb975081d9e82e864" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 44423 } } peers { permanent_uuid: "914e3b58d0c643db820d993c9bee73d5" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 37211 } } peers { permanent_uuid: "ad4d550a1abc4b94a458f42ba15d8112" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 44595 } }
I20260923 23:18:09.663399 12066 raft_consensus.cc:374] T 7063a475533148c5b74a8f3602f0a7e8 P b4ca6c55eb5e498bb975081d9e82e864 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:18:09.663441 12066 raft_consensus.cc:729] T 7063a475533148c5b74a8f3602f0a7e8 P b4ca6c55eb5e498bb975081d9e82e864 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4ca6c55eb5e498bb975081d9e82e864, State: Initialized, Role: FOLLOWER
I20260923 23:18:09.663584 12066 consensus_queue.cc:260] T 7063a475533148c5b74a8f3602f0a7e8 P b4ca6c55eb5e498bb975081d9e82e864 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4ca6c55eb5e498bb975081d9e82e864" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 44423 } } peers { permanent_uuid: "914e3b58d0c643db820d993c9bee73d5" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 37211 } } peers { permanent_uuid: "ad4d550a1abc4b94a458f42ba15d8112" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 44595 } }
I20260923 23:18:09.663830 12066 ts_tablet_manager.cc:1428] T 7063a475533148c5b74a8f3602f0a7e8 P b4ca6c55eb5e498bb975081d9e82e864: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260923 23:18:09.663978 12051 heartbeater.cc:503] Master 127.24.227.254:42709 was elected leader, sending a full tablet report...
I20260923 23:18:09.664165 12067 raft_consensus.cc:348] T 7063a475533148c5b74a8f3602f0a7e8 P ad4d550a1abc4b94a458f42ba15d8112 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4ca6c55eb5e498bb975081d9e82e864" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 44423 } } peers { permanent_uuid: "914e3b58d0c643db820d993c9bee73d5" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 37211 } } peers { permanent_uuid: "ad4d550a1abc4b94a458f42ba15d8112" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 44595 } }
I20260923 23:18:09.664332 12067 raft_consensus.cc:374] T 7063a475533148c5b74a8f3602f0a7e8 P ad4d550a1abc4b94a458f42ba15d8112 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:18:09.664371 12067 raft_consensus.cc:729] T 7063a475533148c5b74a8f3602f0a7e8 P ad4d550a1abc4b94a458f42ba15d8112 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad4d550a1abc4b94a458f42ba15d8112, State: Initialized, Role: FOLLOWER
I20260923 23:18:09.664474 12067 consensus_queue.cc:260] T 7063a475533148c5b74a8f3602f0a7e8 P ad4d550a1abc4b94a458f42ba15d8112 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4ca6c55eb5e498bb975081d9e82e864" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 44423 } } peers { permanent_uuid: "914e3b58d0c643db820d993c9bee73d5" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 37211 } } peers { permanent_uuid: "ad4d550a1abc4b94a458f42ba15d8112" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 44595 } }
I20260923 23:18:09.664710 12067 ts_tablet_manager.cc:1428] T 7063a475533148c5b74a8f3602f0a7e8 P ad4d550a1abc4b94a458f42ba15d8112: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260923 23:18:09.664846 11789 heartbeater.cc:503] Master 127.24.227.254:42709 was elected leader, sending a full tablet report...
I20260923 23:18:09.665437 12065 raft_consensus.cc:348] T 7063a475533148c5b74a8f3602f0a7e8 P 914e3b58d0c643db820d993c9bee73d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4ca6c55eb5e498bb975081d9e82e864" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 44423 } } peers { permanent_uuid: "914e3b58d0c643db820d993c9bee73d5" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 37211 } } peers { permanent_uuid: "ad4d550a1abc4b94a458f42ba15d8112" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 44595 } }
I20260923 23:18:09.665582 12065 raft_consensus.cc:374] T 7063a475533148c5b74a8f3602f0a7e8 P 914e3b58d0c643db820d993c9bee73d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:18:09.665622 12065 raft_consensus.cc:729] T 7063a475533148c5b74a8f3602f0a7e8 P 914e3b58d0c643db820d993c9bee73d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 914e3b58d0c643db820d993c9bee73d5, State: Initialized, Role: FOLLOWER
I20260923 23:18:09.665779 12065 consensus_queue.cc:260] T 7063a475533148c5b74a8f3602f0a7e8 P 914e3b58d0c643db820d993c9bee73d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4ca6c55eb5e498bb975081d9e82e864" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 44423 } } peers { permanent_uuid: "914e3b58d0c643db820d993c9bee73d5" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 37211 } } peers { permanent_uuid: "ad4d550a1abc4b94a458f42ba15d8112" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 44595 } }
I20260923 23:18:09.666052 12065 ts_tablet_manager.cc:1428] T 7063a475533148c5b74a8f3602f0a7e8 P 914e3b58d0c643db820d993c9bee73d5: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 23:18:09.666085 11920 heartbeater.cc:503] Master 127.24.227.254:42709 was elected leader, sending a full tablet report...
W20260923 23:18:09.759765 11921 tablet.cc:2369] T 7063a475533148c5b74a8f3602f0a7e8 P 914e3b58d0c643db820d993c9bee73d5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 23:18:09.877171 12052 tablet.cc:2369] T 7063a475533148c5b74a8f3602f0a7e8 P b4ca6c55eb5e498bb975081d9e82e864: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 23:18:09.883463 11790 tablet.cc:2369] T 7063a475533148c5b74a8f3602f0a7e8 P ad4d550a1abc4b94a458f42ba15d8112: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 23:18:09.885571 12073 raft_consensus.cc:482] T 7063a475533148c5b74a8f3602f0a7e8 P 914e3b58d0c643db820d993c9bee73d5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 23:18:09.885707 12073 raft_consensus.cc:504] T 7063a475533148c5b74a8f3602f0a7e8 P 914e3b58d0c643db820d993c9bee73d5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4ca6c55eb5e498bb975081d9e82e864" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 44423 } } peers { permanent_uuid: "914e3b58d0c643db820d993c9bee73d5" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 37211 } } peers { permanent_uuid: "ad4d550a1abc4b94a458f42ba15d8112" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 44595 } }
I20260923 23:18:09.886040 12073 leader_election.cc:288] T 7063a475533148c5b74a8f3602f0a7e8 P 914e3b58d0c643db820d993c9bee73d5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b4ca6c55eb5e498bb975081d9e82e864 (127.24.227.195:44423), ad4d550a1abc4b94a458f42ba15d8112 (127.24.227.193:44595)
I20260923 23:18:09.888881 11922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:09.886205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.193:44595 (local address 127.24.227.194:50761)
0923 23:18:09.886264 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:09.886276 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 23:18:09.886332 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:18:09.886552 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:18:09.886553 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:18:09.886554 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:18:09.886627 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:09.886629 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:09.887242 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:09.887243 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:09.888068 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:09.888071 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:09.888634 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:09.888635 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:09.888708 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:09.888712 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 23:18:09.888763 (+ 51us) client_negotiation.cc:240] Negotiation successful
0923 23:18:09.888809 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260923 23:18:09.888901 12064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:09.886322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.194:50761 (local address 127.24.227.193:44595)
0923 23:18:09.886410 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:09.886418 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:09.886424 (+ 6us) server_negotiation.cc:407] Connection header received
0923 23:18:09.886468 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:09.886469 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:09.886480 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:09.886549 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:18:09.886708 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:09.887164 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:09.888152 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:09.888551 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:09.888625 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:09.888639 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:09.888783 (+ 144us) server_negotiation.cc:299] Negotiation successful
0923 23:18:09.888824 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260923 23:18:09.889016 12061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:09.886317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.194:49123 (local address 127.24.227.195:44423)
0923 23:18:09.886363 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:09.886365 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:09.886478 (+ 113us) server_negotiation.cc:407] Connection header received
0923 23:18:09.886556 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:09.886557 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:09.886567 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:09.886611 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:18:09.886770 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:09.887230 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:09.888285 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:09.888644 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:09.888713 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:09.888720 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:09.888892 (+ 172us) server_negotiation.cc:299] Negotiation successful
0923 23:18:09.888937 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260923 23:18:09.889168 12074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:09.886205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.195:44423 (local address 127.24.227.194:49123)
0923 23:18:09.886405 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:09.886414 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 23:18:09.886490 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:18:09.886624 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:18:09.886625 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:18:09.886626 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:18:09.886698 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:09.886699 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:09.887311 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:09.887312 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:09.888203 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:09.888205 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:09.888726 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:09.888727 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:09.888815 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:09.888819 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 23:18:09.888872 (+ 53us) client_negotiation.cc:240] Negotiation successful
0923 23:18:09.888905 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"mutex_wait_us":50,"thread_start_us":97,"threads_started":1}
I20260923 23:18:09.889209 11743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7063a475533148c5b74a8f3602f0a7e8" candidate_uuid: "914e3b58d0c643db820d993c9bee73d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad4d550a1abc4b94a458f42ba15d8112" is_pre_election: true
I20260923 23:18:09.889371 11743 raft_consensus.cc:2454] T 7063a475533148c5b74a8f3602f0a7e8 P ad4d550a1abc4b94a458f42ba15d8112 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 914e3b58d0c643db820d993c9bee73d5 in term 0.
I20260923 23:18:09.889480 12005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7063a475533148c5b74a8f3602f0a7e8" candidate_uuid: "914e3b58d0c643db820d993c9bee73d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4ca6c55eb5e498bb975081d9e82e864" is_pre_election: true
I20260923 23:18:09.889571 11810 leader_election.cc:302] T 7063a475533148c5b74a8f3602f0a7e8 P 914e3b58d0c643db820d993c9bee73d5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 914e3b58d0c643db820d993c9bee73d5, ad4d550a1abc4b94a458f42ba15d8112; no voters:
I20260923 23:18:09.889624 12005 raft_consensus.cc:2454] T 7063a475533148c5b74a8f3602f0a7e8 P b4ca6c55eb5e498bb975081d9e82e864 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 914e3b58d0c643db820d993c9bee73d5 in term 0.
I20260923 23:18:09.889709 12073 raft_consensus.cc:2781] T 7063a475533148c5b74a8f3602f0a7e8 P 914e3b58d0c643db820d993c9bee73d5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 23:18:09.889771 12073 raft_consensus.cc:482] T 7063a475533148c5b74a8f3602f0a7e8 P 914e3b58d0c643db820d993c9bee73d5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 23:18:09.889804 12073 raft_consensus.cc:3037] T 7063a475533148c5b74a8f3602f0a7e8 P 914e3b58d0c643db820d993c9bee73d5 [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:18:09.890534 12073 raft_consensus.cc:504] T 7063a475533148c5b74a8f3602f0a7e8 P 914e3b58d0c643db820d993c9bee73d5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4ca6c55eb5e498bb975081d9e82e864" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 44423 } } peers { permanent_uuid: "914e3b58d0c643db820d993c9bee73d5" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 37211 } } peers { permanent_uuid: "ad4d550a1abc4b94a458f42ba15d8112" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 44595 } }
I20260923 23:18:09.890673 12073 leader_election.cc:288] T 7063a475533148c5b74a8f3602f0a7e8 P 914e3b58d0c643db820d993c9bee73d5 [CANDIDATE]: Term 1 election: Requested vote from peers b4ca6c55eb5e498bb975081d9e82e864 (127.24.227.195:44423), ad4d550a1abc4b94a458f42ba15d8112 (127.24.227.193:44595)
I20260923 23:18:09.890862 12005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7063a475533148c5b74a8f3602f0a7e8" candidate_uuid: "914e3b58d0c643db820d993c9bee73d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4ca6c55eb5e498bb975081d9e82e864"
I20260923 23:18:09.890861 11743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7063a475533148c5b74a8f3602f0a7e8" candidate_uuid: "914e3b58d0c643db820d993c9bee73d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad4d550a1abc4b94a458f42ba15d8112"
I20260923 23:18:09.890940 11743 raft_consensus.cc:3037] T 7063a475533148c5b74a8f3602f0a7e8 P ad4d550a1abc4b94a458f42ba15d8112 [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:18:09.890940 12005 raft_consensus.cc:3037] T 7063a475533148c5b74a8f3602f0a7e8 P b4ca6c55eb5e498bb975081d9e82e864 [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:18:09.891808 11743 raft_consensus.cc:2454] T 7063a475533148c5b74a8f3602f0a7e8 P ad4d550a1abc4b94a458f42ba15d8112 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 914e3b58d0c643db820d993c9bee73d5 in term 1.
I20260923 23:18:09.891808 12005 raft_consensus.cc:2454] T 7063a475533148c5b74a8f3602f0a7e8 P b4ca6c55eb5e498bb975081d9e82e864 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 914e3b58d0c643db820d993c9bee73d5 in term 1.
I20260923 23:18:09.891986 11807 leader_election.cc:302] T 7063a475533148c5b74a8f3602f0a7e8 P 914e3b58d0c643db820d993c9bee73d5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 914e3b58d0c643db820d993c9bee73d5, b4ca6c55eb5e498bb975081d9e82e864; no voters:
I20260923 23:18:09.892071 12073 raft_consensus.cc:2781] T 7063a475533148c5b74a8f3602f0a7e8 P 914e3b58d0c643db820d993c9bee73d5 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 23:18:09.892210 12073 raft_consensus.cc:686] T 7063a475533148c5b74a8f3602f0a7e8 P 914e3b58d0c643db820d993c9bee73d5 [term 1 LEADER]: Becoming Leader. State: Replica: 914e3b58d0c643db820d993c9bee73d5, State: Running, Role: LEADER
I20260923 23:18:09.892313 12073 consensus_queue.cc:237] T 7063a475533148c5b74a8f3602f0a7e8 P 914e3b58d0c643db820d993c9bee73d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4ca6c55eb5e498bb975081d9e82e864" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 44423 } } peers { permanent_uuid: "914e3b58d0c643db820d993c9bee73d5" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 37211 } } peers { permanent_uuid: "ad4d550a1abc4b94a458f42ba15d8112" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 44595 } }
I20260923 23:18:09.893108 11600 catalog_manager.cc:5495] T 7063a475533148c5b74a8f3602f0a7e8 P 914e3b58d0c643db820d993c9bee73d5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 914e3b58d0c643db820d993c9bee73d5 (127.24.227.194). New cstate: current_term: 1 leader_uuid: "914e3b58d0c643db820d993c9bee73d5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4ca6c55eb5e498bb975081d9e82e864" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 44423 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "914e3b58d0c643db820d993c9bee73d5" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 37211 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ad4d550a1abc4b94a458f42ba15d8112" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 44595 } health_report { overall_health: UNKNOWN } } }
I20260923 23:18:09.934526 12062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:09.932426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56812 (local address 127.24.227.194:37211)
0923 23:18:09.932485 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:09.932487 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:09.932495 (+ 8us) server_negotiation.cc:407] Connection header received
0923 23:18:09.932545 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:09.932546 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:09.932563 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:09.932618 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 23:18:09.932723 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:09.933156 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:09.933844 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:09.934020 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:09.934063 (+ 43us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:09.934180 (+ 117us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 23:18:09.934256 (+ 76us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 23:18:09.934332 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:09.934412 (+ 80us) server_negotiation.cc:299] Negotiation successful
0923 23:18:09.934421 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260923 23:18:09.936112 12005 raft_consensus.cc:1261] T 7063a475533148c5b74a8f3602f0a7e8 P b4ca6c55eb5e498bb975081d9e82e864 [term 1 FOLLOWER]: Refusing update from remote peer 914e3b58d0c643db820d993c9bee73d5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 23:18:09.936161 11743 raft_consensus.cc:1261] T 7063a475533148c5b74a8f3602f0a7e8 P ad4d550a1abc4b94a458f42ba15d8112 [term 1 FOLLOWER]: Refusing update from remote peer 914e3b58d0c643db820d993c9bee73d5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 23:18:09.936439 12073 consensus_queue.cc:1035] T 7063a475533148c5b74a8f3602f0a7e8 P 914e3b58d0c643db820d993c9bee73d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4ca6c55eb5e498bb975081d9e82e864" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 44423 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 23:18:09.936589 12075 consensus_queue.cc:1035] T 7063a475533148c5b74a8f3602f0a7e8 P 914e3b58d0c643db820d993c9bee73d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad4d550a1abc4b94a458f42ba15d8112" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 44595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 23:18:09.938522 12078 mvcc.cc:205] Tried to move back new op lower bound from 7332681686776012800 to 7332681686599606272. Current Snapshot: MvccSnapshot[applied={T|T < 7332681686776012800}]
I20260923 23:18:09.938906 12080 mvcc.cc:205] Tried to move back new op lower bound from 7332681686776012800 to 7332681686599606272. Current Snapshot: MvccSnapshot[applied={T|T < 7332681686776012800}]
I20260923 23:18:09.938917 12079 mvcc.cc:205] Tried to move back new op lower bound from 7332681686776012800 to 7332681686599606272. Current Snapshot: MvccSnapshot[applied={T|T < 7332681686776012800}]
I20260923 23:18:09.941102 11601 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38084:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 23:18:09.941206 11601 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38084:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 23:18:09.941926 11601 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 21e710c992484e0eab85f4a3a0a9789d: Sending DeleteTablet for 3 replicas of tablet 7063a475533148c5b74a8f3602f0a7e8
I20260923 23:18:09.942276 11985 tablet_service.cc:1514] Processing DeleteTablet for tablet 7063a475533148c5b74a8f3602f0a7e8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 23:18:09 UTC) from {username='slave'} at 127.0.0.1:58486
I20260923 23:18:09.942301 11854 tablet_service.cc:1514] Processing DeleteTablet for tablet 7063a475533148c5b74a8f3602f0a7e8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 23:18:09 UTC) from {username='slave'} at 127.0.0.1:56802
I20260923 23:18:09.942505 12088 tablet_replica.cc:330] T 7063a475533148c5b74a8f3602f0a7e8 P b4ca6c55eb5e498bb975081d9e82e864: stopping tablet replica
I20260923 23:18:09.942495 11723 tablet_service.cc:1514] Processing DeleteTablet for tablet 7063a475533148c5b74a8f3602f0a7e8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 23:18:09 UTC) from {username='slave'} at 127.0.0.1:60772
I20260923 23:18:09.942598 12088 raft_consensus.cc:2229] T 7063a475533148c5b74a8f3602f0a7e8 P b4ca6c55eb5e498bb975081d9e82e864 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 23:18:09.942698 12088 raft_consensus.cc:2258] T 7063a475533148c5b74a8f3602f0a7e8 P b4ca6c55eb5e498bb975081d9e82e864 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 23:18:09.942726 12090 tablet_replica.cc:330] T 7063a475533148c5b74a8f3602f0a7e8 P ad4d550a1abc4b94a458f42ba15d8112: stopping tablet replica
I20260923 23:18:09.942919 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 11661
I20260923 23:18:09.943028 12090 raft_consensus.cc:2229] T 7063a475533148c5b74a8f3602f0a7e8 P ad4d550a1abc4b94a458f42ba15d8112 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 23:18:09.943111 12088 ts_tablet_manager.cc:1905] T 7063a475533148c5b74a8f3602f0a7e8 P b4ca6c55eb5e498bb975081d9e82e864: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 23:18:09.943261 12090 raft_consensus.cc:2258] T 7063a475533148c5b74a8f3602f0a7e8 P ad4d550a1abc4b94a458f42ba15d8112 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 23:18:09.944941 12088 ts_tablet_manager.cc:1918] T 7063a475533148c5b74a8f3602f0a7e8 P b4ca6c55eb5e498bb975081d9e82e864: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 23:18:09.945013 12088 log.cc:1196] T 7063a475533148c5b74a8f3602f0a7e8 P b4ca6c55eb5e498bb975081d9e82e864: Deleting WAL directory at /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790205259943518-25487-0/minicluster-data/ts-2/wal/wals/7063a475533148c5b74a8f3602f0a7e8
I20260923 23:18:09.944957 12089 tablet_replica.cc:330] T 7063a475533148c5b74a8f3602f0a7e8 P 914e3b58d0c643db820d993c9bee73d5: stopping tablet replica
I20260923 23:18:09.945199 12089 raft_consensus.cc:2229] T 7063a475533148c5b74a8f3602f0a7e8 P 914e3b58d0c643db820d993c9bee73d5 [term 1 LEADER]: Raft consensus shutting down.
I20260923 23:18:09.945255 12088 ts_tablet_manager.cc:1939] T 7063a475533148c5b74a8f3602f0a7e8 P b4ca6c55eb5e498bb975081d9e82e864: Deleting consensus metadata
I20260923 23:18:09.945405 12089 raft_consensus.cc:2258] T 7063a475533148c5b74a8f3602f0a7e8 P 914e3b58d0c643db820d993c9bee73d5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 23:18:09.945878 11585 catalog_manager.cc:4841] TS b4ca6c55eb5e498bb975081d9e82e864 (127.24.227.195:44423): tablet 7063a475533148c5b74a8f3602f0a7e8 (table test-table [id=7e000f6c74024f03b8d6519f016e629a]) successfully deleted
I20260923 23:18:09.946290 12089 ts_tablet_manager.cc:1905] T 7063a475533148c5b74a8f3602f0a7e8 P 914e3b58d0c643db820d993c9bee73d5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 23:18:09.948014 12089 ts_tablet_manager.cc:1918] T 7063a475533148c5b74a8f3602f0a7e8 P 914e3b58d0c643db820d993c9bee73d5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 23:18:09.948130 12089 log.cc:1196] T 7063a475533148c5b74a8f3602f0a7e8 P 914e3b58d0c643db820d993c9bee73d5: Deleting WAL directory at /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790205259943518-25487-0/minicluster-data/ts-1/wal/wals/7063a475533148c5b74a8f3602f0a7e8
I20260923 23:18:09.948371 12089 ts_tablet_manager.cc:1939] T 7063a475533148c5b74a8f3602f0a7e8 P 914e3b58d0c643db820d993c9bee73d5: Deleting consensus metadata
I20260923 23:18:09.948724 11588 catalog_manager.cc:4841] TS 914e3b58d0c643db820d993c9bee73d5 (127.24.227.194:37211): tablet 7063a475533148c5b74a8f3602f0a7e8 (table test-table [id=7e000f6c74024f03b8d6519f016e629a]) successfully deleted
W20260923 23:18:09.949406 11588 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.227.193:44595 (error 108)
I20260923 23:18:09.949872 12059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:09.949741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.193:44595 (local address 127.0.0.1:60782)
0923 23:18:09.949800 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:09.949839 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.193:44595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260923 23:18:09.949934 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 11792
W20260923 23:18:09.951285 11588 catalog_manager.cc:4584] TS ad4d550a1abc4b94a458f42ba15d8112 (127.24.227.193:44595): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7063a475533148c5b74a8f3602f0a7e8: Network error: Client connection negotiation failed: client connection to 127.24.227.193:44595: connect: Connection refused (error 111)
I20260923 23:18:09.954890 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 11923
I20260923 23:18:09.958938 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 11570
2026-09-23T23:18:09Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (878 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790205259943518-25487-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 23:18:10 dist-test-slave-754n krb5kdc[12094](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 23:18:10 dist-test-slave-754n krb5kdc[12094](info): set up 2 sockets
Sep 23 23:18:10 dist-test-slave-754n krb5kdc[12094](info): commencing operation
krb5kdc: starting...
W20260923 23:18:10.963541 25487 mini_kdc.cc:121] Time spent starting KDC: real 0.986s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790205259943518-25487-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790205259943518-25487-0/krb5kdc/test-user.keytab.
Sep 23 23:18:11 dist-test-slave-754n krb5kdc[12094](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205491, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T23:18:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T23:18:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.227.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.227.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.227.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab.
Entry for principal kudu/127.24.227.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab.
I20260923 23:18:11.072582 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790205259943518-25487-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790205259943518-25487-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790205259943518-25487-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.254:33897
--webserver_interface=127.24.227.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab
--principal=kudu/127.24.227.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:33349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.227.254:33897
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:18:11.147742 12107 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:18:11.147976 12107 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:18:11.148020 12107 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:18:11.149101 12107 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 23:18:11.149161 12107 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:18:11.149184 12107 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:18:11.149205 12107 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 23:18:11.149251 12107 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 23:18:11.150715 12107 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:33349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790205259943518-25487-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790205259943518-25487-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.227.254:33897
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab
--principal=kudu/127.24.227.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.254:33897
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.227.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12107
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790205259943518-25487-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:18:11.150961 12107 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:18:11.151206 12107 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:18:11.155642 12115 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:18:11.155654 12112 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:18:11.155679 12113 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:18:11.155973 12107 server_base.cc:1034] running on GCE node
I20260923 23:18:11.156234 12107 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:18:11.156507 12107 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:18:11.157661 12107 hybrid_clock.cc:648] HybridClock initialized: now 1790205491157647 us; error 44 us; skew 500 ppm
Sep 23 23:18:11 dist-test-slave-754n krb5kdc[12094](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205491, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:18:11.159855 12107 init.cc:377] Logged in from keytab as kudu/127.24.227.254@KRBTEST.COM (short username kudu)
I20260923 23:18:11.160346 12107 webserver.cc:460] Webserver started at http://127.24.227.254:33257/ using document root <none> and password file <none>
I20260923 23:18:11.160566 12107 fs_manager.cc:359] Metadata directory not provided
I20260923 23:18:11.160629 12107 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:18:11.160763 12107 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:18:11.161731 12107 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790205259943518-25487-0/minicluster-data/master-0/data/instance:
uuid: "d5cf600d41ae4188929137f85fc0a828"
format_stamp: "Formatted at 2026-09-23 23:18:11 on dist-test-slave-754n"
server_key: "607b229634207757a8a0d7d2f3bae6c8"
server_key_iv: "fa658f95113d5190284d21e777f51426"
server_key_version: "encryptionkey@0"
I20260923 23:18:11.162077 12107 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790205259943518-25487-0/minicluster-data/master-0/wal/instance:
uuid: "d5cf600d41ae4188929137f85fc0a828"
format_stamp: "Formatted at 2026-09-23 23:18:11 on dist-test-slave-754n"
server_key: "607b229634207757a8a0d7d2f3bae6c8"
server_key_iv: "fa658f95113d5190284d21e777f51426"
server_key_version: "encryptionkey@0"
I20260923 23:18:11.163344 12107 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 23:18:11.164057 12122 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:18:11.164232 12107 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 23:18:11.164336 12107 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790205259943518-25487-0/minicluster-data/master-0/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790205259943518-25487-0/minicluster-data/master-0/wal
uuid: "d5cf600d41ae4188929137f85fc0a828"
format_stamp: "Formatted at 2026-09-23 23:18:11 on dist-test-slave-754n"
server_key: "607b229634207757a8a0d7d2f3bae6c8"
server_key_iv: "fa658f95113d5190284d21e777f51426"
server_key_version: "encryptionkey@0"
I20260923 23:18:11.164419 12107 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790205259943518-25487-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790205259943518-25487-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790205259943518-25487-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:18:11.180956 12107 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:18:11.182600 12107 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:18:11.182754 12107 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:18:11.186479 12107 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.254:33897
I20260923 23:18:11.186529 12174 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.254:33897 every 8 connection(s)
I20260923 23:18:11.186864 12107 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
I20260923 23:18:11.187613 12175 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:18:11.189697 12175 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d5cf600d41ae4188929137f85fc0a828: Bootstrap starting.
I20260923 23:18:11.190246 12175 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d5cf600d41ae4188929137f85fc0a828: Neither blocks nor log segments found. Creating new log.
I20260923 23:18:11.190522 12175 log.cc:824] T 00000000000000000000000000000000 P d5cf600d41ae4188929137f85fc0a828: Log is configured to *not* fsync() on all Append() calls
I20260923 23:18:11.191164 12175 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d5cf600d41ae4188929137f85fc0a828: No bootstrap required, opened a new log
I20260923 23:18:11.192759 12175 raft_consensus.cc:348] T 00000000000000000000000000000000 P d5cf600d41ae4188929137f85fc0a828 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5cf600d41ae4188929137f85fc0a828" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 33897 } }
I20260923 23:18:11.192876 12175 raft_consensus.cc:374] T 00000000000000000000000000000000 P d5cf600d41ae4188929137f85fc0a828 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:18:11.192898 12175 raft_consensus.cc:729] T 00000000000000000000000000000000 P d5cf600d41ae4188929137f85fc0a828 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5cf600d41ae4188929137f85fc0a828, State: Initialized, Role: FOLLOWER
I20260923 23:18:11.193054 12175 consensus_queue.cc:260] T 00000000000000000000000000000000 P d5cf600d41ae4188929137f85fc0a828 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5cf600d41ae4188929137f85fc0a828" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 33897 } }
I20260923 23:18:11.193111 12175 raft_consensus.cc:388] T 00000000000000000000000000000000 P d5cf600d41ae4188929137f85fc0a828 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 23:18:11.193154 12175 raft_consensus.cc:482] T 00000000000000000000000000000000 P d5cf600d41ae4188929137f85fc0a828 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 23:18:11.193202 12175 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d5cf600d41ae4188929137f85fc0a828 [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:18:11.193673 12175 raft_consensus.cc:504] T 00000000000000000000000000000000 P d5cf600d41ae4188929137f85fc0a828 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5cf600d41ae4188929137f85fc0a828" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 33897 } }
I20260923 23:18:11.193775 12175 leader_election.cc:302] T 00000000000000000000000000000000 P d5cf600d41ae4188929137f85fc0a828 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d5cf600d41ae4188929137f85fc0a828; no voters:
I20260923 23:18:11.193953 12175 leader_election.cc:288] T 00000000000000000000000000000000 P d5cf600d41ae4188929137f85fc0a828 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 23:18:11.193995 12178 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d5cf600d41ae4188929137f85fc0a828 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 23:18:11.194156 12178 raft_consensus.cc:686] T 00000000000000000000000000000000 P d5cf600d41ae4188929137f85fc0a828 [term 1 LEADER]: Becoming Leader. State: Replica: d5cf600d41ae4188929137f85fc0a828, State: Running, Role: LEADER
I20260923 23:18:11.194185 12175 sys_catalog.cc:564] T 00000000000000000000000000000000 P d5cf600d41ae4188929137f85fc0a828 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 23:18:11.194236 12178 consensus_queue.cc:237] T 00000000000000000000000000000000 P d5cf600d41ae4188929137f85fc0a828 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5cf600d41ae4188929137f85fc0a828" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 33897 } }
I20260923 23:18:11.194617 12179 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5cf600d41ae4188929137f85fc0a828 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d5cf600d41ae4188929137f85fc0a828" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5cf600d41ae4188929137f85fc0a828" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 33897 } } }
I20260923 23:18:11.194710 12179 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5cf600d41ae4188929137f85fc0a828 [sys.catalog]: This master's current role is: LEADER
I20260923 23:18:11.194725 12180 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5cf600d41ae4188929137f85fc0a828 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d5cf600d41ae4188929137f85fc0a828. Latest consensus state: current_term: 1 leader_uuid: "d5cf600d41ae4188929137f85fc0a828" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5cf600d41ae4188929137f85fc0a828" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 33897 } } }
I20260923 23:18:11.194775 12180 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5cf600d41ae4188929137f85fc0a828 [sys.catalog]: This master's current role is: LEADER
I20260923 23:18:11.194927 12187 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 23:18:11.195437 12187 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 23:18:11.196461 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 12107
I20260923 23:18:11.196604 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790205259943518-25487-0/minicluster-data/master-0/wal/instance
I20260923 23:18:11.196826 25487 external_mini_cluster.cc:1428] Setting key 4a5108bc1e0a5d7d828afdf8d990cce2
I20260923 23:18:11.196879 12187 catalog_manager.cc:1347] Generated new cluster ID: e757d3b477014b36a03c2da873838cd4
I20260923 23:18:11.196931 12187 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 23:18:11 dist-test-slave-754n krb5kdc[12094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205491, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
I20260923 23:18:11.205812 12198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:11.197225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58698 (local address 127.24.227.254:33897)
0923 23:18:11.197491 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:11.197499 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:11.197525 (+ 26us) server_negotiation.cc:407] Connection header received
0923 23:18:11.197628 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:11.197633 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:11.197671 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:11.197795 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:11.198361 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:11.199235 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:11.200000 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:11.200409 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:11.200541 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:11.202738 (+ 2197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:18:11.202741 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:18:11.202742 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:18:11.202753 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:18:11.204391 (+ 1638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:11.204739 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:11.204742 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:11.204742 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:11.204839 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:11.205041 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:11.205042 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:11.205042 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:11.205260 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:18:11.205426 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:11.205487 (+ 61us) server_negotiation.cc:299] Negotiation successful
0923 23:18:11.205619 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":232,"thread_start_us":73,"threads_started":1}
I20260923 23:18:11.224458 12187 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 23:18:11.225118 12187 catalog_manager.cc:1504] Loading token signing keys...
I20260923 23:18:11.235232 12187 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d5cf600d41ae4188929137f85fc0a828: Generated new TSK 0
I20260923 23:18:11.235385 12187 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.227.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.227.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.227.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.193.keytab.
Entry for principal kudu/127.24.227.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.193.keytab.
I20260923 23:18:11.264390 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790205259943518-25487-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790205259943518-25487-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790205259943518-25487-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790205259943518-25487-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.227.193:0
--local_ip_for_outbound_sockets=127.24.227.193
--webserver_interface=127.24.227.193
--webserver_port=0
--tserver_master_addrs=127.24.227.254:33897
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.193.keytab
--principal=kudu/127.24.227.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:33349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:18:11.343176 12201 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:18:11.343354 12201 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:18:11.343375 12201 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:18:11.344458 12201 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:18:11.344494 12201 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:18:11.344518 12201 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.227.193
I20260923 23:18:11.345846 12201 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:33349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790205259943518-25487-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790205259943518-25487-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.193.keytab
--principal=kudu/127.24.227.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.227.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790205259943518-25487-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.227.193
--webserver_port=0
--tserver_master_addrs=127.24.227.254:33897
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12201
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.227.193
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790205259943518-25487-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:18:11.346089 12201 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:18:11.346310 12201 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260923 23:18:11.351061 12201 server_base.cc:1034] running on GCE node
W20260923 23:18:11.351034 12209 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:18:11.351045 12207 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:18:11.351045 12206 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:18:11.351406 12201 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:18:11.351656 12201 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:18:11.352813 12201 hybrid_clock.cc:648] HybridClock initialized: now 1790205491352805 us; error 40 us; skew 500 ppm
Sep 23 23:18:11 dist-test-slave-754n krb5kdc[12094](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205491, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:18:11.355003 12201 init.cc:377] Logged in from keytab as kudu/127.24.227.193@KRBTEST.COM (short username kudu)
I20260923 23:18:11.355470 12201 webserver.cc:460] Webserver started at http://127.24.227.193:35797/ using document root <none> and password file <none>
I20260923 23:18:11.355690 12201 fs_manager.cc:359] Metadata directory not provided
I20260923 23:18:11.355753 12201 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:18:11.355909 12201 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:18:11.356870 12201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790205259943518-25487-0/minicluster-data/ts-0/data/instance:
uuid: "b2c84869f6184752b11de95c97890861"
format_stamp: "Formatted at 2026-09-23 23:18:11 on dist-test-slave-754n"
server_key: "2414d14197d871765ffbadd63360cacc"
server_key_iv: "0fa8aaf4da41d8dcc267d06ebff9d47e"
server_key_version: "encryptionkey@0"
I20260923 23:18:11.357203 12201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790205259943518-25487-0/minicluster-data/ts-0/wal/instance:
uuid: "b2c84869f6184752b11de95c97890861"
format_stamp: "Formatted at 2026-09-23 23:18:11 on dist-test-slave-754n"
server_key: "2414d14197d871765ffbadd63360cacc"
server_key_iv: "0fa8aaf4da41d8dcc267d06ebff9d47e"
server_key_version: "encryptionkey@0"
I20260923 23:18:11.358414 12201 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260923 23:18:11.359123 12216 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:18:11.359287 12201 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 23:18:11.359354 12201 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790205259943518-25487-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790205259943518-25487-0/minicluster-data/ts-0/wal
uuid: "b2c84869f6184752b11de95c97890861"
format_stamp: "Formatted at 2026-09-23 23:18:11 on dist-test-slave-754n"
server_key: "2414d14197d871765ffbadd63360cacc"
server_key_iv: "0fa8aaf4da41d8dcc267d06ebff9d47e"
server_key_version: "encryptionkey@0"
I20260923 23:18:11.359436 12201 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790205259943518-25487-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790205259943518-25487-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790205259943518-25487-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:18:11.372880 12201 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:18:11.374944 12201 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:18:11.375059 12201 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:18:11.375262 12201 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 23:18:11.375535 12201 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 23:18:11.375567 12201 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 23:18:11.375607 12201 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 23:18:11.375631 12201 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 23:18:11.381340 12201 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.193:38281
I20260923 23:18:11.381400 12329 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.193:38281 every 8 connection(s)
I20260923 23:18:11.381687 12201 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790205259943518-25487-0/minicluster-data/ts-0/data/info.pb
Sep 23 23:18:11 dist-test-slave-754n krb5kdc[12094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205491, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.193@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
I20260923 23:18:11.388769 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 12201
I20260923 23:18:11.388886 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790205259943518-25487-0/minicluster-data/ts-0/wal/instance
I20260923 23:18:11.389104 25487 external_mini_cluster.cc:1428] Setting key 0e3efb6bbdf25b5c75d187fc194ae0e6
I20260923 23:18:11.390435 12198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:11.382352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.193:43699 (local address 127.24.227.254:33897)
0923 23:18:11.382406 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:11.382407 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:11.382601 (+ 194us) server_negotiation.cc:407] Connection header received
0923 23:18:11.383073 (+ 472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:11.383074 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:11.383090 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:11.383143 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:11.384059 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:11.384561 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:11.385551 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:11.385764 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:11.385839 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:11.387690 (+ 1851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:18:11.387692 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:18:11.387692 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:18:11.387699 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:18:11.389062 (+ 1363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:11.389475 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:11.389475 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:11.389476 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:11.389524 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:11.389787 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:11.389787 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:11.389788 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:11.389912 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:18:11.389964 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:11.390310 (+ 346us) server_negotiation.cc:299] Negotiation successful
0923 23:18:11.390375 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260923 23:18:11.390708 12332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:11.382321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:33897 (local address 127.24.227.193:43699)
0923 23:18:11.382486 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:11.382525 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0923 23:18:11.382974 (+ 449us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:18:11.383208 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:18:11.383211 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:18:11.383724 (+ 513us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:18:11.383958 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:11.383963 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:11.384664 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:11.384665 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:11.385451 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:11.385453 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:11.385844 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:11.385846 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:11.385980 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:11.386372 (+ 392us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:18:11.386379 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:18:11.387595 (+ 1216us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:18:11.389151 (+ 1556us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:18:11.389172 (+ 21us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:18:11.389174 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:18:11.389389 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:18:11.389583 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:18:11.389584 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:18:11.389584 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:18:11.389712 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:18:11.389982 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:18:11.389988 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:18:11.390170 (+ 182us) client_negotiation.cc:769] Sending connection context
0923 23:18:11.390291 (+ 121us) client_negotiation.cc:240] Negotiation successful
0923 23:18:11.390456 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":79,"threads_started":1}
I20260923 23:18:11.391145 12330 heartbeater.cc:347] Connected to a master server at 127.24.227.254:33897
I20260923 23:18:11.391296 12330 heartbeater.cc:464] Registering TS with master...
I20260923 23:18:11.391481 12330 heartbeater.cc:511] Master 127.24.227.254:33897 requested a full tablet report, sending...
I20260923 23:18:11.392033 12139 ts_manager.cc:194] Registered new tserver with Master: b2c84869f6184752b11de95c97890861 (127.24.227.193:38281)
I20260923 23:18:11.392949 12139 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.227.193@KRBTEST.COM'} at 127.24.227.193:43699
WARNING: no policy specified for kudu/127.24.227.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.227.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.227.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.194.keytab.
Entry for principal kudu/127.24.227.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.194.keytab.
I20260923 23:18:11.412632 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790205259943518-25487-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790205259943518-25487-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790205259943518-25487-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790205259943518-25487-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.227.194:0
--local_ip_for_outbound_sockets=127.24.227.194
--webserver_interface=127.24.227.194
--webserver_port=0
--tserver_master_addrs=127.24.227.254:33897
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.194.keytab
--principal=kudu/127.24.227.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:33349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:18:11.491722 12335 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:18:11.491971 12335 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:18:11.491998 12335 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:18:11.493139 12335 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:18:11.493176 12335 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:18:11.493228 12335 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.227.194
I20260923 23:18:11.494890 12335 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:33349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790205259943518-25487-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790205259943518-25487-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.194.keytab
--principal=kudu/127.24.227.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.227.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790205259943518-25487-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.227.194
--webserver_port=0
--tserver_master_addrs=127.24.227.254:33897
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12335
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.227.194
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790205259943518-25487-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:18:11.495139 12335 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:18:11.495388 12335 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:18:11.500077 12340 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:18:11.500129 12335 server_base.cc:1034] running on GCE node
W20260923 23:18:11.500058 12341 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:18:11.500248 12343 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:18:11.500444 12335 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:18:11.500720 12335 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:18:11.501875 12335 hybrid_clock.cc:648] HybridClock initialized: now 1790205491501853 us; error 52 us; skew 500 ppm
Sep 23 23:18:11 dist-test-slave-754n krb5kdc[12094](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205491, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:18:11.504132 12335 init.cc:377] Logged in from keytab as kudu/127.24.227.194@KRBTEST.COM (short username kudu)
I20260923 23:18:11.504546 12335 webserver.cc:460] Webserver started at http://127.24.227.194:35471/ using document root <none> and password file <none>
I20260923 23:18:11.504750 12335 fs_manager.cc:359] Metadata directory not provided
I20260923 23:18:11.504809 12335 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:18:11.504930 12335 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:18:11.505846 12335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790205259943518-25487-0/minicluster-data/ts-1/data/instance:
uuid: "e8d0776c21b343d3b333fdab543a0765"
format_stamp: "Formatted at 2026-09-23 23:18:11 on dist-test-slave-754n"
server_key: "9b1739bd1f7af1e23f820ae1439779d8"
server_key_iv: "0dccf4d1add1a2bfe4d78683706b9765"
server_key_version: "encryptionkey@0"
I20260923 23:18:11.506206 12335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790205259943518-25487-0/minicluster-data/ts-1/wal/instance:
uuid: "e8d0776c21b343d3b333fdab543a0765"
format_stamp: "Formatted at 2026-09-23 23:18:11 on dist-test-slave-754n"
server_key: "9b1739bd1f7af1e23f820ae1439779d8"
server_key_iv: "0dccf4d1add1a2bfe4d78683706b9765"
server_key_version: "encryptionkey@0"
I20260923 23:18:11.507409 12335 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 23:18:11.508153 12350 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:18:11.508309 12335 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 23:18:11.508399 12335 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790205259943518-25487-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790205259943518-25487-0/minicluster-data/ts-1/wal
uuid: "e8d0776c21b343d3b333fdab543a0765"
format_stamp: "Formatted at 2026-09-23 23:18:11 on dist-test-slave-754n"
server_key: "9b1739bd1f7af1e23f820ae1439779d8"
server_key_iv: "0dccf4d1add1a2bfe4d78683706b9765"
server_key_version: "encryptionkey@0"
I20260923 23:18:11.508477 12335 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790205259943518-25487-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790205259943518-25487-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790205259943518-25487-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:18:11.518658 12335 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:18:11.520412 12335 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:18:11.520555 12335 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:18:11.520753 12335 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 23:18:11.521021 12335 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 23:18:11.521075 12335 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 23:18:11.521123 12335 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 23:18:11.521154 12335 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 23:18:11.526649 12335 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.194:39453
I20260923 23:18:11.526695 12463 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.194:39453 every 8 connection(s)
I20260923 23:18:11.526937 12335 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790205259943518-25487-0/minicluster-data/ts-1/data/info.pb
Sep 23 23:18:11 dist-test-slave-754n krb5kdc[12094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205491, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.194@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
I20260923 23:18:11.534848 12198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:11.527537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.194:35931 (local address 127.24.227.254:33897)
0923 23:18:11.527618 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:11.527621 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:11.527789 (+ 168us) server_negotiation.cc:407] Connection header received
0923 23:18:11.528255 (+ 466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:11.528256 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:11.528266 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:11.528316 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:11.529101 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:11.529570 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:11.530492 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:11.530684 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:11.530739 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:11.532662 (+ 1923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:18:11.532664 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:18:11.532664 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:18:11.532671 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:18:11.533796 (+ 1125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:11.534146 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:11.534147 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:11.534147 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:11.534194 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:11.534384 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:11.534384 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:11.534384 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:11.534474 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:18:11.534520 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:11.534721 (+ 201us) server_negotiation.cc:299] Negotiation successful
0923 23:18:11.534778 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260923 23:18:11.535007 12466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:11.527502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:33897 (local address 127.24.227.194:35931)
0923 23:18:11.527680 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:11.527711 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0923 23:18:11.528175 (+ 464us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:18:11.528360 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:18:11.528363 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:18:11.528861 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:18:11.529026 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:11.529029 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:11.529644 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:11.529645 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:11.530416 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:11.530417 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:11.530752 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:11.530753 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:11.530843 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:11.531231 (+ 388us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:18:11.531239 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:18:11.532574 (+ 1335us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:18:11.533861 (+ 1287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:18:11.533867 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:18:11.533868 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:18:11.534085 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:18:11.534248 (+ 163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:18:11.534249 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:18:11.534249 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:18:11.534336 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:18:11.534524 (+ 188us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:18:11.534527 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:18:11.534625 (+ 98us) client_negotiation.cc:769] Sending connection context
0923 23:18:11.534699 (+ 74us) client_negotiation.cc:240] Negotiation successful
0923 23:18:11.534799 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":111,"threads_started":1}
I20260923 23:18:11.535497 12464 heartbeater.cc:347] Connected to a master server at 127.24.227.254:33897
I20260923 23:18:11.535584 12464 heartbeater.cc:464] Registering TS with master...
I20260923 23:18:11.535737 12464 heartbeater.cc:511] Master 127.24.227.254:33897 requested a full tablet report, sending...
I20260923 23:18:11.536135 12139 ts_manager.cc:194] Registered new tserver with Master: e8d0776c21b343d3b333fdab543a0765 (127.24.227.194:39453)
I20260923 23:18:11.536446 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 12335
I20260923 23:18:11.536509 12139 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.227.194@KRBTEST.COM'} at 127.24.227.194:35931
I20260923 23:18:11.536543 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790205259943518-25487-0/minicluster-data/ts-1/wal/instance
I20260923 23:18:11.536795 25487 external_mini_cluster.cc:1428] Setting key b13d13973550dbc815a820cb69bd53f2
WARNING: no policy specified for kudu/127.24.227.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.227.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.227.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.195.keytab.
Entry for principal kudu/127.24.227.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.195.keytab.
I20260923 23:18:11.559098 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790205259943518-25487-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790205259943518-25487-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790205259943518-25487-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790205259943518-25487-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.227.195:0
--local_ip_for_outbound_sockets=127.24.227.195
--webserver_interface=127.24.227.195
--webserver_port=0
--tserver_master_addrs=127.24.227.254:33897
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.195.keytab
--principal=kudu/127.24.227.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:33349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:18:11.637205 12469 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:18:11.637379 12469 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:18:11.637400 12469 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:18:11.638618 12469 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:18:11.638659 12469 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:18:11.638684 12469 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.227.195
I20260923 23:18:11.640183 12469 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:33349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790205259943518-25487-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790205259943518-25487-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.195.keytab
--principal=kudu/127.24.227.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.227.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790205259943518-25487-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.227.195
--webserver_port=0
--tserver_master_addrs=127.24.227.254:33897
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12469
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.227.195
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790205259943518-25487-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:18:11.640408 12469 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:18:11.640631 12469 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:18:11.645391 12474 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:18:11.645391 12477 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:18:11.645592 12469 server_base.cc:1034] running on GCE node
W20260923 23:18:11.645639 12475 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:18:11.645838 12469 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:18:11.646049 12469 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:18:11.647197 12469 hybrid_clock.cc:648] HybridClock initialized: now 1790205491647187 us; error 25 us; skew 500 ppm
Sep 23 23:18:11 dist-test-slave-754n krb5kdc[12094](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205491, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:18:11.649617 12469 init.cc:377] Logged in from keytab as kudu/127.24.227.195@KRBTEST.COM (short username kudu)
I20260923 23:18:11.650051 12469 webserver.cc:460] Webserver started at http://127.24.227.195:40483/ using document root <none> and password file <none>
I20260923 23:18:11.650303 12469 fs_manager.cc:359] Metadata directory not provided
I20260923 23:18:11.650369 12469 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:18:11.650516 12469 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:18:11.651490 12469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790205259943518-25487-0/minicluster-data/ts-2/data/instance:
uuid: "f89963479fbf4a8d9aed71afba827704"
format_stamp: "Formatted at 2026-09-23 23:18:11 on dist-test-slave-754n"
server_key: "7fe5e0a6095c7c3b28a9067a83f86072"
server_key_iv: "903c122866fd43e4cac45a874124c3d6"
server_key_version: "encryptionkey@0"
I20260923 23:18:11.651839 12469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790205259943518-25487-0/minicluster-data/ts-2/wal/instance:
uuid: "f89963479fbf4a8d9aed71afba827704"
format_stamp: "Formatted at 2026-09-23 23:18:11 on dist-test-slave-754n"
server_key: "7fe5e0a6095c7c3b28a9067a83f86072"
server_key_iv: "903c122866fd43e4cac45a874124c3d6"
server_key_version: "encryptionkey@0"
I20260923 23:18:11.653095 12469 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 23:18:11.653803 12484 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:18:11.653970 12469 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 23:18:11.654090 12469 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790205259943518-25487-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790205259943518-25487-0/minicluster-data/ts-2/wal
uuid: "f89963479fbf4a8d9aed71afba827704"
format_stamp: "Formatted at 2026-09-23 23:18:11 on dist-test-slave-754n"
server_key: "7fe5e0a6095c7c3b28a9067a83f86072"
server_key_iv: "903c122866fd43e4cac45a874124c3d6"
server_key_version: "encryptionkey@0"
I20260923 23:18:11.654161 12469 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790205259943518-25487-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790205259943518-25487-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790205259943518-25487-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:18:11.679000 12469 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:18:11.680781 12469 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:18:11.680932 12469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:18:11.681147 12469 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 23:18:11.681442 12469 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 23:18:11.681496 12469 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 23:18:11.681537 12469 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 23:18:11.681564 12469 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 23:18:11.687282 12469 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.195:35161
I20260923 23:18:11.687330 12597 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.195:35161 every 8 connection(s)
I20260923 23:18:11.687644 12469 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790205259943518-25487-0/minicluster-data/ts-2/data/info.pb
Sep 23 23:18:11 dist-test-slave-754n krb5kdc[12094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205491, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.195@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
I20260923 23:18:11.693140 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 12469
I20260923 23:18:11.693248 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790205259943518-25487-0/minicluster-data/ts-2/wal/instance
I20260923 23:18:11.693444 25487 external_mini_cluster.cc:1428] Setting key 55cfca8c2376561102832c50a9d24a58
I20260923 23:18:11.696585 12198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:11.688315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.195:59433 (local address 127.24.227.254:33897)
0923 23:18:11.688367 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:11.688369 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:11.688568 (+ 199us) server_negotiation.cc:407] Connection header received
0923 23:18:11.688984 (+ 416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:11.688985 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:11.688995 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:11.689042 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:11.689888 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:11.690380 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:11.691292 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:11.691473 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:11.691525 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:11.693645 (+ 2120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:18:11.693647 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:18:11.693647 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:18:11.693654 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:18:11.695486 (+ 1832us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:11.695832 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:11.695833 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:11.695834 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:11.695908 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:11.696115 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:11.696115 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:11.696116 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:11.696211 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:18:11.696260 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:11.696470 (+ 210us) server_negotiation.cc:299] Negotiation successful
0923 23:18:11.696527 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260923 23:18:11.696734 12600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:11.688295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:33897 (local address 127.24.227.195:59433)
0923 23:18:11.688455 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:11.688487 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0923 23:18:11.688905 (+ 418us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:18:11.689100 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:18:11.689103 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:18:11.689637 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:18:11.689813 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:11.689817 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:11.690456 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:11.690457 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:11.691226 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:11.691228 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:11.691545 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:11.691546 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:11.691638 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:11.692094 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:18:11.692102 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:18:11.693559 (+ 1457us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:18:11.695551 (+ 1992us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:18:11.695556 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:18:11.695557 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:18:11.695762 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:18:11.695973 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:18:11.695974 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:18:11.695974 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:18:11.696059 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:18:11.696265 (+ 206us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:18:11.696270 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:18:11.696368 (+ 98us) client_negotiation.cc:769] Sending connection context
0923 23:18:11.696444 (+ 76us) client_negotiation.cc:240] Negotiation successful
0923 23:18:11.696547 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":82,"threads_started":1}
I20260923 23:18:11.697290 12598 heartbeater.cc:347] Connected to a master server at 127.24.227.254:33897
I20260923 23:18:11.697402 12598 heartbeater.cc:464] Registering TS with master...
I20260923 23:18:11.697559 12598 heartbeater.cc:511] Master 127.24.227.254:33897 requested a full tablet report, sending...
I20260923 23:18:11.697916 12139 ts_manager.cc:194] Registered new tserver with Master: f89963479fbf4a8d9aed71afba827704 (127.24.227.195:35161)
I20260923 23:18:11.698264 12139 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.227.195@KRBTEST.COM'} at 127.24.227.195:59433
I20260923 23:18:11.705281 25487 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 23:18:11.711781 12198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:11.706546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58714 (local address 127.24.227.254:33897)
0923 23:18:11.706585 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:11.706587 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:11.706693 (+ 106us) server_negotiation.cc:407] Connection header received
0923 23:18:11.706734 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:11.706735 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:11.706743 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:11.706790 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:11.707256 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:11.707674 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:11.708392 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:11.708554 (+ 162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:11.708625 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:11.709579 (+ 954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:18:11.709580 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:18:11.709580 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:18:11.709585 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:18:11.710944 (+ 1359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:11.711232 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:11.711233 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:11.711233 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:11.711281 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:11.711394 (+ 113us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:11.711394 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:11.711394 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:11.711504 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:18:11.711564 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:11.711644 (+ 80us) server_negotiation.cc:299] Negotiation successful
0923 23:18:11.711694 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260923 23:18:11.713069 12139 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58714:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 23:18:11.714568 12139 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 23:18:11.720480 12607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:11.716621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.193:38281 (local address 127.0.0.1:44582)
0923 23:18:11.716740 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:11.716782 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0923 23:18:11.716844 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:18:11.717241 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:18:11.717243 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:18:11.717246 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:18:11.717303 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:11.717305 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:11.718236 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:11.718237 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:11.719329 (+ 1092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:11.719332 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:11.720207 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:11.720208 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:11.720281 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:11.720293 (+ 12us) client_negotiation.cc:769] Sending connection context
0923 23:18:11.720361 (+ 68us) client_negotiation.cc:240] Negotiation successful
0923 23:18:11.720410 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95,"mutex_wait_us":7,"thread_start_us":44,"threads_started":1}
I20260923 23:18:11.720492 12610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:11.716809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44582 (local address 127.24.227.193:38281)
0923 23:18:11.717060 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:11.717066 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:11.717076 (+ 10us) server_negotiation.cc:407] Connection header received
0923 23:18:11.717119 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:11.717121 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:11.717138 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:11.717219 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:18:11.717384 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:11.718133 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:11.719426 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:11.720121 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:11.720207 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:11.720261 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:11.720375 (+ 114us) server_negotiation.cc:299] Negotiation successful
0923 23:18:11.720417 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":214,"thread_start_us":84,"threads_started":1}
I20260923 23:18:11.720780 12606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:11.716797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.194:39453 (local address 127.0.0.1:48180)
0923 23:18:11.716976 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:11.716994 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0923 23:18:11.717028 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:18:11.717442 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:18:11.717443 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:18:11.717444 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:18:11.717480 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:11.717481 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:11.718741 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:11.718742 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:11.719642 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:11.719644 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:11.720554 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:11.720555 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:11.720646 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:11.720652 (+ 6us) client_negotiation.cc:769] Sending connection context
0923 23:18:11.720680 (+ 28us) client_negotiation.cc:240] Negotiation successful
0923 23:18:11.720716 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":46,"threads_started":1}
I20260923 23:18:11.721036 12609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:11.716636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.195:35161 (local address 127.0.0.1:50140)
0923 23:18:11.716875 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:11.716886 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 23:18:11.716937 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:18:11.717423 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:18:11.717424 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:18:11.717427 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:18:11.717471 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:11.717472 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:11.718712 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:11.718713 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:11.719777 (+ 1064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:11.719779 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:11.720862 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:11.720863 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:11.720914 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:11.720917 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 23:18:11.720945 (+ 28us) client_negotiation.cc:240] Negotiation successful
0923 23:18:11.720982 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":61,"threads_started":1}
I20260923 23:18:11.721089 12611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:11.716978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48180 (local address 127.24.227.194:39453)
0923 23:18:11.717167 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:11.717172 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:11.717181 (+ 9us) server_negotiation.cc:407] Connection header received
0923 23:18:11.717218 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:11.717220 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:11.717347 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:11.717549 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:18:11.717631 (+ 82us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:11.718663 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:11.719726 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:11.720448 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:11.720884 (+ 436us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:11.720948 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:11.720972 (+ 24us) server_negotiation.cc:299] Negotiation successful
0923 23:18:11.721009 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":89,"threads_started":1}
I20260923 23:18:11.721210 12264 tablet_service.cc:1467] Processing CreateTablet for tablet 5efdcb310a764634aa69744e7dfc5a9f (DEFAULT_TABLE table=test-table [id=509f1ca793254b1eae2cc23835a875dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 23:18:11.721422 12608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:11.716721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50140 (local address 127.24.227.195:35161)
0923 23:18:11.717167 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:11.717172 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:11.717181 (+ 9us) server_negotiation.cc:407] Connection header received
0923 23:18:11.717217 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:11.717219 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:11.717235 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:11.717341 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:18:11.717547 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:11.718606 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:11.719888 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:11.720510 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:11.721255 (+ 745us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:11.721294 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:11.721313 (+ 19us) server_negotiation.cc:299] Negotiation successful
0923 23:18:11.721342 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":418,"thread_start_us":70,"threads_started":1}
I20260923 23:18:11.721602 12264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5efdcb310a764634aa69744e7dfc5a9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:18:11.722049 12532 tablet_service.cc:1467] Processing CreateTablet for tablet 5efdcb310a764634aa69744e7dfc5a9f (DEFAULT_TABLE table=test-table [id=509f1ca793254b1eae2cc23835a875dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 23:18:11.722229 12398 tablet_service.cc:1467] Processing CreateTablet for tablet 5efdcb310a764634aa69744e7dfc5a9f (DEFAULT_TABLE table=test-table [id=509f1ca793254b1eae2cc23835a875dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 23:18:11.722347 12532 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5efdcb310a764634aa69744e7dfc5a9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:18:11.722554 12398 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5efdcb310a764634aa69744e7dfc5a9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:18:11.724997 12612 tablet_bootstrap.cc:492] T 5efdcb310a764634aa69744e7dfc5a9f P b2c84869f6184752b11de95c97890861: Bootstrap starting.
I20260923 23:18:11.725301 12613 tablet_bootstrap.cc:492] T 5efdcb310a764634aa69744e7dfc5a9f P e8d0776c21b343d3b333fdab543a0765: Bootstrap starting.
I20260923 23:18:11.725564 12614 tablet_bootstrap.cc:492] T 5efdcb310a764634aa69744e7dfc5a9f P f89963479fbf4a8d9aed71afba827704: Bootstrap starting.
I20260923 23:18:11.725824 12612 tablet_bootstrap.cc:654] T 5efdcb310a764634aa69744e7dfc5a9f P b2c84869f6184752b11de95c97890861: Neither blocks nor log segments found. Creating new log.
I20260923 23:18:11.725924 12613 tablet_bootstrap.cc:654] T 5efdcb310a764634aa69744e7dfc5a9f P e8d0776c21b343d3b333fdab543a0765: Neither blocks nor log segments found. Creating new log.
I20260923 23:18:11.726280 12614 tablet_bootstrap.cc:654] T 5efdcb310a764634aa69744e7dfc5a9f P f89963479fbf4a8d9aed71afba827704: Neither blocks nor log segments found. Creating new log.
I20260923 23:18:11.726423 12612 log.cc:824] T 5efdcb310a764634aa69744e7dfc5a9f P b2c84869f6184752b11de95c97890861: Log is configured to *not* fsync() on all Append() calls
I20260923 23:18:11.726435 12613 log.cc:824] T 5efdcb310a764634aa69744e7dfc5a9f P e8d0776c21b343d3b333fdab543a0765: Log is configured to *not* fsync() on all Append() calls
I20260923 23:18:11.726836 12614 log.cc:824] T 5efdcb310a764634aa69744e7dfc5a9f P f89963479fbf4a8d9aed71afba827704: Log is configured to *not* fsync() on all Append() calls
I20260923 23:18:11.727185 12613 tablet_bootstrap.cc:492] T 5efdcb310a764634aa69744e7dfc5a9f P e8d0776c21b343d3b333fdab543a0765: No bootstrap required, opened a new log
I20260923 23:18:11.727258 12612 tablet_bootstrap.cc:492] T 5efdcb310a764634aa69744e7dfc5a9f P b2c84869f6184752b11de95c97890861: No bootstrap required, opened a new log
I20260923 23:18:11.727260 12613 ts_tablet_manager.cc:1397] T 5efdcb310a764634aa69744e7dfc5a9f P e8d0776c21b343d3b333fdab543a0765: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 23:18:11.727326 12612 ts_tablet_manager.cc:1397] T 5efdcb310a764634aa69744e7dfc5a9f P b2c84869f6184752b11de95c97890861: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260923 23:18:11.727676 12614 tablet_bootstrap.cc:492] T 5efdcb310a764634aa69744e7dfc5a9f P f89963479fbf4a8d9aed71afba827704: No bootstrap required, opened a new log
I20260923 23:18:11.727741 12614 ts_tablet_manager.cc:1397] T 5efdcb310a764634aa69744e7dfc5a9f P f89963479fbf4a8d9aed71afba827704: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260923 23:18:11.728856 12613 raft_consensus.cc:348] T 5efdcb310a764634aa69744e7dfc5a9f P e8d0776c21b343d3b333fdab543a0765 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2c84869f6184752b11de95c97890861" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 38281 } } peers { permanent_uuid: "f89963479fbf4a8d9aed71afba827704" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 35161 } } peers { permanent_uuid: "e8d0776c21b343d3b333fdab543a0765" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 39453 } }
I20260923 23:18:11.729008 12613 raft_consensus.cc:374] T 5efdcb310a764634aa69744e7dfc5a9f P e8d0776c21b343d3b333fdab543a0765 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:18:11.729050 12613 raft_consensus.cc:729] T 5efdcb310a764634aa69744e7dfc5a9f P e8d0776c21b343d3b333fdab543a0765 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8d0776c21b343d3b333fdab543a0765, State: Initialized, Role: FOLLOWER
I20260923 23:18:11.729154 12613 consensus_queue.cc:260] T 5efdcb310a764634aa69744e7dfc5a9f P e8d0776c21b343d3b333fdab543a0765 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2c84869f6184752b11de95c97890861" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 38281 } } peers { permanent_uuid: "f89963479fbf4a8d9aed71afba827704" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 35161 } } peers { permanent_uuid: "e8d0776c21b343d3b333fdab543a0765" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 39453 } }
I20260923 23:18:11.729409 12613 ts_tablet_manager.cc:1428] T 5efdcb310a764634aa69744e7dfc5a9f P e8d0776c21b343d3b333fdab543a0765: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260923 23:18:11.729305 12612 raft_consensus.cc:348] T 5efdcb310a764634aa69744e7dfc5a9f P b2c84869f6184752b11de95c97890861 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2c84869f6184752b11de95c97890861" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 38281 } } peers { permanent_uuid: "f89963479fbf4a8d9aed71afba827704" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 35161 } } peers { permanent_uuid: "e8d0776c21b343d3b333fdab543a0765" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 39453 } }
I20260923 23:18:11.729453 12612 raft_consensus.cc:374] T 5efdcb310a764634aa69744e7dfc5a9f P b2c84869f6184752b11de95c97890861 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:18:11.729496 12612 raft_consensus.cc:729] T 5efdcb310a764634aa69744e7dfc5a9f P b2c84869f6184752b11de95c97890861 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2c84869f6184752b11de95c97890861, State: Initialized, Role: FOLLOWER
I20260923 23:18:11.729559 12464 heartbeater.cc:503] Master 127.24.227.254:33897 was elected leader, sending a full tablet report...
I20260923 23:18:11.729609 12612 consensus_queue.cc:260] T 5efdcb310a764634aa69744e7dfc5a9f P b2c84869f6184752b11de95c97890861 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2c84869f6184752b11de95c97890861" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 38281 } } peers { permanent_uuid: "f89963479fbf4a8d9aed71afba827704" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 35161 } } peers { permanent_uuid: "e8d0776c21b343d3b333fdab543a0765" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 39453 } }
I20260923 23:18:11.729565 12614 raft_consensus.cc:348] T 5efdcb310a764634aa69744e7dfc5a9f P f89963479fbf4a8d9aed71afba827704 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2c84869f6184752b11de95c97890861" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 38281 } } peers { permanent_uuid: "f89963479fbf4a8d9aed71afba827704" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 35161 } } peers { permanent_uuid: "e8d0776c21b343d3b333fdab543a0765" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 39453 } }
I20260923 23:18:11.729720 12614 raft_consensus.cc:374] T 5efdcb310a764634aa69744e7dfc5a9f P f89963479fbf4a8d9aed71afba827704 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:18:11.729768 12614 raft_consensus.cc:729] T 5efdcb310a764634aa69744e7dfc5a9f P f89963479fbf4a8d9aed71afba827704 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f89963479fbf4a8d9aed71afba827704, State: Initialized, Role: FOLLOWER
I20260923 23:18:11.729861 12612 ts_tablet_manager.cc:1428] T 5efdcb310a764634aa69744e7dfc5a9f P b2c84869f6184752b11de95c97890861: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260923 23:18:11.729861 12614 consensus_queue.cc:260] T 5efdcb310a764634aa69744e7dfc5a9f P f89963479fbf4a8d9aed71afba827704 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2c84869f6184752b11de95c97890861" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 38281 } } peers { permanent_uuid: "f89963479fbf4a8d9aed71afba827704" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 35161 } } peers { permanent_uuid: "e8d0776c21b343d3b333fdab543a0765" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 39453 } }
I20260923 23:18:11.729981 12330 heartbeater.cc:503] Master 127.24.227.254:33897 was elected leader, sending a full tablet report...
I20260923 23:18:11.730136 12614 ts_tablet_manager.cc:1428] T 5efdcb310a764634aa69744e7dfc5a9f P f89963479fbf4a8d9aed71afba827704: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260923 23:18:11.730154 12598 heartbeater.cc:503] Master 127.24.227.254:33897 was elected leader, sending a full tablet report...
W20260923 23:18:11.777407 12465 tablet.cc:2369] T 5efdcb310a764634aa69744e7dfc5a9f P e8d0776c21b343d3b333fdab543a0765: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 23:18:11.882351 12331 tablet.cc:2369] T 5efdcb310a764634aa69744e7dfc5a9f P b2c84869f6184752b11de95c97890861: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 23:18:11.938184 12599 tablet.cc:2369] T 5efdcb310a764634aa69744e7dfc5a9f P f89963479fbf4a8d9aed71afba827704: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 23:18:11.998265 12618 raft_consensus.cc:482] T 5efdcb310a764634aa69744e7dfc5a9f P e8d0776c21b343d3b333fdab543a0765 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 23:18:11.998407 12618 raft_consensus.cc:504] T 5efdcb310a764634aa69744e7dfc5a9f P e8d0776c21b343d3b333fdab543a0765 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2c84869f6184752b11de95c97890861" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 38281 } } peers { permanent_uuid: "f89963479fbf4a8d9aed71afba827704" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 35161 } } peers { permanent_uuid: "e8d0776c21b343d3b333fdab543a0765" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 39453 } }
I20260923 23:18:11.998797 12618 leader_election.cc:288] T 5efdcb310a764634aa69744e7dfc5a9f P e8d0776c21b343d3b333fdab543a0765 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b2c84869f6184752b11de95c97890861 (127.24.227.193:38281), f89963479fbf4a8d9aed71afba827704 (127.24.227.195:35161)
I20260923 23:18:12.001420 12466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:11.998936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.193:38281 (local address 127.24.227.194:50659)
0923 23:18:11.998987 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:11.998998 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 23:18:11.999038 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:18:11.999196 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:18:11.999197 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:18:11.999198 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:18:11.999273 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:11.999275 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:11.999922 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:11.999923 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:12.000630 (+ 707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:12.000632 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:12.001139 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:12.001140 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:12.001241 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:12.001244 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 23:18:12.001310 (+ 66us) client_negotiation.cc:240] Negotiation successful
0923 23:18:12.001349 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32,"mutex_wait_us":5}
I20260923 23:18:12.001417 12610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:11.999005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.194:50659 (local address 127.24.227.193:38281)
0923 23:18:11.999043 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:11.999046 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:11.999054 (+ 8us) server_negotiation.cc:407] Connection header received
0923 23:18:11.999120 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:11.999121 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:11.999130 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:11.999186 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:18:11.999344 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:11.999810 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:12.000722 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:12.001061 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:12.001143 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:12.001151 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:12.001316 (+ 165us) server_negotiation.cc:299] Negotiation successful
0923 23:18:12.001353 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 23:18:12.001605 12621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:11.998953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.195:35161 (local address 127.24.227.194:40583)
0923 23:18:11.999116 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:11.999126 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 23:18:11.999206 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:18:11.999351 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:18:11.999352 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:18:11.999354 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:18:11.999425 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:11.999427 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:12.000083 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:12.000084 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:12.000785 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:12.000787 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:12.001344 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:12.001345 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:12.001423 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:12.001425 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 23:18:12.001486 (+ 61us) client_negotiation.cc:240] Negotiation successful
0923 23:18:12.001527 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":90,"threads_started":1}
I20260923 23:18:12.001605 12608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:11.999064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.194:40583 (local address 127.24.227.195:35161)
0923 23:18:11.999108 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:11.999110 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:11.999195 (+ 85us) server_negotiation.cc:407] Connection header received
0923 23:18:11.999268 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:11.999268 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:11.999290 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:11.999346 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:18:11.999494 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:11.999992 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:12.000874 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:12.001258 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:12.001343 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:12.001351 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:12.001504 (+ 153us) server_negotiation.cc:299] Negotiation successful
0923 23:18:12.001541 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 23:18:12.001750 12284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5efdcb310a764634aa69744e7dfc5a9f" candidate_uuid: "e8d0776c21b343d3b333fdab543a0765" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2c84869f6184752b11de95c97890861" is_pre_election: true
I20260923 23:18:12.001869 12552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5efdcb310a764634aa69744e7dfc5a9f" candidate_uuid: "e8d0776c21b343d3b333fdab543a0765" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f89963479fbf4a8d9aed71afba827704" is_pre_election: true
I20260923 23:18:12.001917 12284 raft_consensus.cc:2454] T 5efdcb310a764634aa69744e7dfc5a9f P b2c84869f6184752b11de95c97890861 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8d0776c21b343d3b333fdab543a0765 in term 0.
I20260923 23:18:12.002002 12552 raft_consensus.cc:2454] T 5efdcb310a764634aa69744e7dfc5a9f P f89963479fbf4a8d9aed71afba827704 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8d0776c21b343d3b333fdab543a0765 in term 0.
I20260923 23:18:12.002179 12354 leader_election.cc:302] T 5efdcb310a764634aa69744e7dfc5a9f P e8d0776c21b343d3b333fdab543a0765 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2c84869f6184752b11de95c97890861, e8d0776c21b343d3b333fdab543a0765; no voters:
I20260923 23:18:12.002305 12618 raft_consensus.cc:2781] T 5efdcb310a764634aa69744e7dfc5a9f P e8d0776c21b343d3b333fdab543a0765 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 23:18:12.002358 12618 raft_consensus.cc:482] T 5efdcb310a764634aa69744e7dfc5a9f P e8d0776c21b343d3b333fdab543a0765 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 23:18:12.002391 12618 raft_consensus.cc:3037] T 5efdcb310a764634aa69744e7dfc5a9f P e8d0776c21b343d3b333fdab543a0765 [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:18:12.003060 12618 raft_consensus.cc:504] T 5efdcb310a764634aa69744e7dfc5a9f P e8d0776c21b343d3b333fdab543a0765 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2c84869f6184752b11de95c97890861" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 38281 } } peers { permanent_uuid: "f89963479fbf4a8d9aed71afba827704" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 35161 } } peers { permanent_uuid: "e8d0776c21b343d3b333fdab543a0765" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 39453 } }
I20260923 23:18:12.003196 12618 leader_election.cc:288] T 5efdcb310a764634aa69744e7dfc5a9f P e8d0776c21b343d3b333fdab543a0765 [CANDIDATE]: Term 1 election: Requested vote from peers b2c84869f6184752b11de95c97890861 (127.24.227.193:38281), f89963479fbf4a8d9aed71afba827704 (127.24.227.195:35161)
I20260923 23:18:12.003355 12552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5efdcb310a764634aa69744e7dfc5a9f" candidate_uuid: "e8d0776c21b343d3b333fdab543a0765" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f89963479fbf4a8d9aed71afba827704"
I20260923 23:18:12.003427 12552 raft_consensus.cc:3037] T 5efdcb310a764634aa69744e7dfc5a9f P f89963479fbf4a8d9aed71afba827704 [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:18:12.004134 12552 raft_consensus.cc:2454] T 5efdcb310a764634aa69744e7dfc5a9f P f89963479fbf4a8d9aed71afba827704 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8d0776c21b343d3b333fdab543a0765 in term 1.
I20260923 23:18:12.004271 12284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5efdcb310a764634aa69744e7dfc5a9f" candidate_uuid: "e8d0776c21b343d3b333fdab543a0765" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2c84869f6184752b11de95c97890861"
I20260923 23:18:12.004304 12352 leader_election.cc:302] T 5efdcb310a764634aa69744e7dfc5a9f P e8d0776c21b343d3b333fdab543a0765 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e8d0776c21b343d3b333fdab543a0765, f89963479fbf4a8d9aed71afba827704; no voters:
I20260923 23:18:12.004355 12284 raft_consensus.cc:3037] T 5efdcb310a764634aa69744e7dfc5a9f P b2c84869f6184752b11de95c97890861 [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:18:12.004457 12618 raft_consensus.cc:2781] T 5efdcb310a764634aa69744e7dfc5a9f P e8d0776c21b343d3b333fdab543a0765 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 23:18:12.004601 12618 raft_consensus.cc:686] T 5efdcb310a764634aa69744e7dfc5a9f P e8d0776c21b343d3b333fdab543a0765 [term 1 LEADER]: Becoming Leader. State: Replica: e8d0776c21b343d3b333fdab543a0765, State: Running, Role: LEADER
I20260923 23:18:12.004689 12618 consensus_queue.cc:237] T 5efdcb310a764634aa69744e7dfc5a9f P e8d0776c21b343d3b333fdab543a0765 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2c84869f6184752b11de95c97890861" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 38281 } } peers { permanent_uuid: "f89963479fbf4a8d9aed71afba827704" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 35161 } } peers { permanent_uuid: "e8d0776c21b343d3b333fdab543a0765" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 39453 } }
I20260923 23:18:12.005266 12284 raft_consensus.cc:2454] T 5efdcb310a764634aa69744e7dfc5a9f P b2c84869f6184752b11de95c97890861 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8d0776c21b343d3b333fdab543a0765 in term 1.
I20260923 23:18:12.005579 12137 catalog_manager.cc:5495] T 5efdcb310a764634aa69744e7dfc5a9f P e8d0776c21b343d3b333fdab543a0765 reported cstate change: term changed from 0 to 1, leader changed from <none> to e8d0776c21b343d3b333fdab543a0765 (127.24.227.194). New cstate: current_term: 1 leader_uuid: "e8d0776c21b343d3b333fdab543a0765" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2c84869f6184752b11de95c97890861" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 38281 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f89963479fbf4a8d9aed71afba827704" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 35161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8d0776c21b343d3b333fdab543a0765" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 39453 } health_report { overall_health: HEALTHY } } }
I20260923 23:18:12.071386 12611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:12.069359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48196 (local address 127.24.227.194:39453)
0923 23:18:12.069412 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:12.069414 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:12.069428 (+ 14us) server_negotiation.cc:407] Connection header received
0923 23:18:12.069458 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:12.069458 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:12.069473 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:12.069529 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 23:18:12.069629 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:12.070062 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:12.070757 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:12.070936 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:12.070984 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:12.071127 (+ 143us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 23:18:12.071220 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 23:18:12.071279 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:12.071314 (+ 35us) server_negotiation.cc:299] Negotiation successful
0923 23:18:12.071330 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260923 23:18:12.073097 12284 raft_consensus.cc:1261] T 5efdcb310a764634aa69744e7dfc5a9f P b2c84869f6184752b11de95c97890861 [term 1 FOLLOWER]: Refusing update from remote peer e8d0776c21b343d3b333fdab543a0765: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 23:18:12.073097 12552 raft_consensus.cc:1261] T 5efdcb310a764634aa69744e7dfc5a9f P f89963479fbf4a8d9aed71afba827704 [term 1 FOLLOWER]: Refusing update from remote peer e8d0776c21b343d3b333fdab543a0765: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 23:18:12.073443 12622 consensus_queue.cc:1035] T 5efdcb310a764634aa69744e7dfc5a9f P e8d0776c21b343d3b333fdab543a0765 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f89963479fbf4a8d9aed71afba827704" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 35161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 23:18:12.073570 12618 consensus_queue.cc:1035] T 5efdcb310a764634aa69744e7dfc5a9f P e8d0776c21b343d3b333fdab543a0765 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2c84869f6184752b11de95c97890861" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 38281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 23:18:12.075929 12626 mvcc.cc:205] Tried to move back new op lower bound from 7332681695528570880 to 7332681695251677184. Current Snapshot: MvccSnapshot[applied={T|T < 7332681695528570880}]
I20260923 23:18:12.076090 12625 mvcc.cc:205] Tried to move back new op lower bound from 7332681695528570880 to 7332681695251677184. Current Snapshot: MvccSnapshot[applied={T|T < 7332681695528570880 or (T in {7332681695528570880})}]
I20260923 23:18:12.076185 12627 mvcc.cc:205] Tried to move back new op lower bound from 7332681695528570880 to 7332681695251677184. Current Snapshot: MvccSnapshot[applied={T|T < 7332681695528570880}]
I20260923 23:18:12.077894 12138 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58714:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 23:18:12.077980 12138 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58714:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 23:18:12.078738 12138 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d5cf600d41ae4188929137f85fc0a828: Sending DeleteTablet for 3 replicas of tablet 5efdcb310a764634aa69744e7dfc5a9f
I20260923 23:18:12.079037 12264 tablet_service.cc:1514] Processing DeleteTablet for tablet 5efdcb310a764634aa69744e7dfc5a9f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 23:18:12 UTC) from {username='kudu', principal='kudu/127.24.227.254@KRBTEST.COM'} at 127.0.0.1:44582
I20260923 23:18:12.079087 12398 tablet_service.cc:1514] Processing DeleteTablet for tablet 5efdcb310a764634aa69744e7dfc5a9f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 23:18:12 UTC) from {username='kudu', principal='kudu/127.24.227.254@KRBTEST.COM'} at 127.0.0.1:48180
I20260923 23:18:12.079245 12532 tablet_service.cc:1514] Processing DeleteTablet for tablet 5efdcb310a764634aa69744e7dfc5a9f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 23:18:12 UTC) from {username='kudu', principal='kudu/127.24.227.254@KRBTEST.COM'} at 127.0.0.1:50140
I20260923 23:18:12.079309 12636 tablet_replica.cc:330] T 5efdcb310a764634aa69744e7dfc5a9f P e8d0776c21b343d3b333fdab543a0765: stopping tablet replica
I20260923 23:18:12.079485 12636 raft_consensus.cc:2229] T 5efdcb310a764634aa69744e7dfc5a9f P e8d0776c21b343d3b333fdab543a0765 [term 1 LEADER]: Raft consensus shutting down.
I20260923 23:18:12.079553 12635 tablet_replica.cc:330] T 5efdcb310a764634aa69744e7dfc5a9f P b2c84869f6184752b11de95c97890861: stopping tablet replica
I20260923 23:18:12.079636 12635 raft_consensus.cc:2229] T 5efdcb310a764634aa69744e7dfc5a9f P b2c84869f6184752b11de95c97890861 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 23:18:12.079653 12636 raft_consensus.cc:2258] T 5efdcb310a764634aa69744e7dfc5a9f P e8d0776c21b343d3b333fdab543a0765 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 23:18:12.079733 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 12201
I20260923 23:18:12.079743 12635 raft_consensus.cc:2258] T 5efdcb310a764634aa69744e7dfc5a9f P b2c84869f6184752b11de95c97890861 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 23:18:12.080108 12636 ts_tablet_manager.cc:1905] T 5efdcb310a764634aa69744e7dfc5a9f P e8d0776c21b343d3b333fdab543a0765: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 23:18:12.081317 12637 tablet_replica.cc:330] T 5efdcb310a764634aa69744e7dfc5a9f P f89963479fbf4a8d9aed71afba827704: stopping tablet replica
I20260923 23:18:12.081432 12637 raft_consensus.cc:2229] T 5efdcb310a764634aa69744e7dfc5a9f P f89963479fbf4a8d9aed71afba827704 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 23:18:12.081526 12637 raft_consensus.cc:2258] T 5efdcb310a764634aa69744e7dfc5a9f P f89963479fbf4a8d9aed71afba827704 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 23:18:12.081645 12636 ts_tablet_manager.cc:1918] T 5efdcb310a764634aa69744e7dfc5a9f P e8d0776c21b343d3b333fdab543a0765: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 23:18:12.081730 12636 log.cc:1196] T 5efdcb310a764634aa69744e7dfc5a9f P e8d0776c21b343d3b333fdab543a0765: Deleting WAL directory at /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790205259943518-25487-0/minicluster-data/ts-1/wal/wals/5efdcb310a764634aa69744e7dfc5a9f
I20260923 23:18:12.081935 12637 ts_tablet_manager.cc:1905] T 5efdcb310a764634aa69744e7dfc5a9f P f89963479fbf4a8d9aed71afba827704: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 23:18:12.082028 12636 ts_tablet_manager.cc:1939] T 5efdcb310a764634aa69744e7dfc5a9f P e8d0776c21b343d3b333fdab543a0765: Deleting consensus metadata
I20260923 23:18:12.082415 12126 catalog_manager.cc:4841] TS e8d0776c21b343d3b333fdab543a0765 (127.24.227.194:39453): tablet 5efdcb310a764634aa69744e7dfc5a9f (table test-table [id=509f1ca793254b1eae2cc23835a875dc]) successfully deleted
I20260923 23:18:12.083292 12637 ts_tablet_manager.cc:1918] T 5efdcb310a764634aa69744e7dfc5a9f P f89963479fbf4a8d9aed71afba827704: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 23:18:12.083386 12637 log.cc:1196] T 5efdcb310a764634aa69744e7dfc5a9f P f89963479fbf4a8d9aed71afba827704: Deleting WAL directory at /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790205259943518-25487-0/minicluster-data/ts-2/wal/wals/5efdcb310a764634aa69744e7dfc5a9f
I20260923 23:18:12.083622 12637 ts_tablet_manager.cc:1939] T 5efdcb310a764634aa69744e7dfc5a9f P f89963479fbf4a8d9aed71afba827704: Deleting consensus metadata
I20260923 23:18:12.084121 12124 catalog_manager.cc:4841] TS f89963479fbf4a8d9aed71afba827704 (127.24.227.195:35161): tablet 5efdcb310a764634aa69744e7dfc5a9f (table test-table [id=509f1ca793254b1eae2cc23835a875dc]) successfully deleted
W20260923 23:18:12.086928 12126 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.227.193:38281 (error 108)
I20260923 23:18:12.087121 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 12335
I20260923 23:18:12.087347 12609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:12.087235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.193:38281 (local address 127.0.0.1:44594)
0923 23:18:12.087276 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:12.087314 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.193:38281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260923 23:18:12.087424 12126 catalog_manager.cc:4584] TS b2c84869f6184752b11de95c97890861 (127.24.227.193:38281): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5efdcb310a764634aa69744e7dfc5a9f: Network error: Client connection negotiation failed: client connection to 127.24.227.193:38281: connect: Connection refused (error 111)
I20260923 23:18:12.091856 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 12469
I20260923 23:18:12.096433 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 12107
2026-09-23T23:18:12Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2138 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-09-23T23:18:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T23:18:12Z Disabled control of system clock
I20260923 23:18:12.123057 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790205259943518-25487-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790205259943518-25487-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790205259943518-25487-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.254:43997
--webserver_interface=127.24.227.254
--webserver_port=0
--builtin_ntp_servers=127.24.227.212:44325
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.227.254:43997
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260923 23:18:12.199401 12642 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:18:12.199585 12642 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:18:12.199604 12642 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:18:12.200675 12642 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 23:18:12.200706 12642 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:18:12.200717 12642 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:18:12.200729 12642 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 23:18:12.200742 12642 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 23:18:12.201954 12642 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:44325
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790205259943518-25487-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790205259943518-25487-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.227.254:43997
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.254:43997
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.227.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12642
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790205259943518-25487-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:18:12.202126 12642 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:18:12.202337 12642 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:18:12.206768 12647 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:18:12.206910 12642 server_base.cc:1034] running on GCE node
W20260923 23:18:12.206755 12650 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:18:12.206744 12648 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:18:12.207249 12642 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:18:12.207494 12642 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:18:12.208623 12642 hybrid_clock.cc:648] HybridClock initialized: now 1790205492208612 us; error 26 us; skew 500 ppm
I20260923 23:18:12.209790 12642 webserver.cc:460] Webserver started at http://127.24.227.254:38503/ using document root <none> and password file <none>
I20260923 23:18:12.209978 12642 fs_manager.cc:359] Metadata directory not provided
I20260923 23:18:12.210037 12642 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:18:12.210120 12642 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:18:12.211021 12642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790205259943518-25487-0/minicluster-data/master-0/data/instance:
uuid: "030bb48fd6104199a923a51c7921cc37"
format_stamp: "Formatted at 2026-09-23 23:18:12 on dist-test-slave-754n"
server_key: "788ecdfffc75c2469a24e002f7bc02bd"
server_key_iv: "8b9f3b1cdfb125c3d7133933d68de7d9"
server_key_version: "encryptionkey@0"
I20260923 23:18:12.211301 12642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790205259943518-25487-0/minicluster-data/master-0/wal/instance:
uuid: "030bb48fd6104199a923a51c7921cc37"
format_stamp: "Formatted at 2026-09-23 23:18:12 on dist-test-slave-754n"
server_key: "788ecdfffc75c2469a24e002f7bc02bd"
server_key_iv: "8b9f3b1cdfb125c3d7133933d68de7d9"
server_key_version: "encryptionkey@0"
I20260923 23:18:12.212531 12642 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 23:18:12.213155 12656 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:18:12.213300 12642 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 23:18:12.213379 12642 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790205259943518-25487-0/minicluster-data/master-0/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790205259943518-25487-0/minicluster-data/master-0/wal
uuid: "030bb48fd6104199a923a51c7921cc37"
format_stamp: "Formatted at 2026-09-23 23:18:12 on dist-test-slave-754n"
server_key: "788ecdfffc75c2469a24e002f7bc02bd"
server_key_iv: "8b9f3b1cdfb125c3d7133933d68de7d9"
server_key_version: "encryptionkey@0"
I20260923 23:18:12.213430 12642 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790205259943518-25487-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790205259943518-25487-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790205259943518-25487-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:18:12.232079 12642 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:18:12.232283 12642 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:18:12.232374 12642 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:18:12.235946 12642 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.254:43997
I20260923 23:18:12.236030 12708 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.254:43997 every 8 connection(s)
I20260923 23:18:12.236313 12642 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
I20260923 23:18:12.236764 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 12642
I20260923 23:18:12.236874 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790205259943518-25487-0/minicluster-data/master-0/wal/instance
I20260923 23:18:12.237046 25487 external_mini_cluster.cc:1428] Setting key 52a4e7d5d65fe86cb00eca28dd962897
I20260923 23:18:12.237396 12709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:18:12.240658 12709 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 030bb48fd6104199a923a51c7921cc37: Bootstrap starting.
I20260923 23:18:12.241333 12709 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 030bb48fd6104199a923a51c7921cc37: Neither blocks nor log segments found. Creating new log.
I20260923 23:18:12.241638 12709 log.cc:824] T 00000000000000000000000000000000 P 030bb48fd6104199a923a51c7921cc37: Log is configured to *not* fsync() on all Append() calls
I20260923 23:18:12.242306 12709 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 030bb48fd6104199a923a51c7921cc37: No bootstrap required, opened a new log
I20260923 23:18:12.243078 12712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:12.237556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35742 (local address 127.24.227.254:43997)
0923 23:18:12.237737 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:12.237745 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:12.237775 (+ 30us) server_negotiation.cc:407] Connection header received
0923 23:18:12.238412 (+ 637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:12.238421 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:12.238511 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:12.238653 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:12.238797 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:12.240184 (+ 1387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:12.240963 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:12.241607 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:12.241732 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:12.242362 (+ 630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:18:12.242366 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:18:12.242366 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 23:18:12.242578 (+ 212us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:18:12.242617 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 23:18:12.242630 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:18:12.242701 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:12.242796 (+ 95us) server_negotiation.cc:299] Negotiation successful
0923 23:18:12.242874 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":83,"threads_started":1}
I20260923 23:18:12.244387 12709 raft_consensus.cc:348] T 00000000000000000000000000000000 P 030bb48fd6104199a923a51c7921cc37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "030bb48fd6104199a923a51c7921cc37" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 43997 } }
I20260923 23:18:12.244506 12709 raft_consensus.cc:374] T 00000000000000000000000000000000 P 030bb48fd6104199a923a51c7921cc37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:18:12.244527 12709 raft_consensus.cc:729] T 00000000000000000000000000000000 P 030bb48fd6104199a923a51c7921cc37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 030bb48fd6104199a923a51c7921cc37, State: Initialized, Role: FOLLOWER
I20260923 23:18:12.244628 12709 consensus_queue.cc:260] T 00000000000000000000000000000000 P 030bb48fd6104199a923a51c7921cc37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "030bb48fd6104199a923a51c7921cc37" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 43997 } }
I20260923 23:18:12.244709 12709 raft_consensus.cc:388] T 00000000000000000000000000000000 P 030bb48fd6104199a923a51c7921cc37 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 23:18:12.244730 12709 raft_consensus.cc:482] T 00000000000000000000000000000000 P 030bb48fd6104199a923a51c7921cc37 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 23:18:12.244757 12709 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 030bb48fd6104199a923a51c7921cc37 [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:18:12.245218 12709 raft_consensus.cc:504] T 00000000000000000000000000000000 P 030bb48fd6104199a923a51c7921cc37 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "030bb48fd6104199a923a51c7921cc37" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 43997 } }
I20260923 23:18:12.245296 12709 leader_election.cc:302] T 00000000000000000000000000000000 P 030bb48fd6104199a923a51c7921cc37 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 030bb48fd6104199a923a51c7921cc37; no voters:
I20260923 23:18:12.245419 12709 leader_election.cc:288] T 00000000000000000000000000000000 P 030bb48fd6104199a923a51c7921cc37 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 23:18:12.245462 12714 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 030bb48fd6104199a923a51c7921cc37 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 23:18:12.245636 12714 raft_consensus.cc:686] T 00000000000000000000000000000000 P 030bb48fd6104199a923a51c7921cc37 [term 1 LEADER]: Becoming Leader. State: Replica: 030bb48fd6104199a923a51c7921cc37, State: Running, Role: LEADER
I20260923 23:18:12.245705 12709 sys_catalog.cc:564] T 00000000000000000000000000000000 P 030bb48fd6104199a923a51c7921cc37 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 23:18:12.245715 12714 consensus_queue.cc:237] T 00000000000000000000000000000000 P 030bb48fd6104199a923a51c7921cc37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "030bb48fd6104199a923a51c7921cc37" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 43997 } }
I20260923 23:18:12.246091 12716 sys_catalog.cc:455] T 00000000000000000000000000000000 P 030bb48fd6104199a923a51c7921cc37 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 030bb48fd6104199a923a51c7921cc37. Latest consensus state: current_term: 1 leader_uuid: "030bb48fd6104199a923a51c7921cc37" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "030bb48fd6104199a923a51c7921cc37" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 43997 } } }
I20260923 23:18:12.246073 12715 sys_catalog.cc:455] T 00000000000000000000000000000000 P 030bb48fd6104199a923a51c7921cc37 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "030bb48fd6104199a923a51c7921cc37" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "030bb48fd6104199a923a51c7921cc37" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 43997 } } }
I20260923 23:18:12.246173 12716 sys_catalog.cc:458] T 00000000000000000000000000000000 P 030bb48fd6104199a923a51c7921cc37 [sys.catalog]: This master's current role is: LEADER
I20260923 23:18:12.246181 12715 sys_catalog.cc:458] T 00000000000000000000000000000000 P 030bb48fd6104199a923a51c7921cc37 [sys.catalog]: This master's current role is: LEADER
I20260923 23:18:12.246421 12721 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 23:18:12.246876 12721 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 23:18:12.248178 12721 catalog_manager.cc:1347] Generated new cluster ID: 1ccc774b1d66484493f7110a84e00bb5
I20260923 23:18:12.248224 12721 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 23:18:12.256747 12721 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 23:18:12.257170 12721 catalog_manager.cc:1504] Loading token signing keys...
I20260923 23:18:12.261268 12721 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 030bb48fd6104199a923a51c7921cc37: Generated new TSK 0
I20260923 23:18:12.261439 12721 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 23:18:12.266654 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790205259943518-25487-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790205259943518-25487-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790205259943518-25487-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790205259943518-25487-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.227.193:0
--local_ip_for_outbound_sockets=127.24.227.193
--webserver_interface=127.24.227.193
--webserver_port=0
--tserver_master_addrs=127.24.227.254:43997
--builtin_ntp_servers=127.24.227.212:44325
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260923 23:18:12.343443 12733 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:18:12.343619 12733 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:18:12.343640 12733 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:18:12.344717 12733 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:18:12.344751 12733 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:18:12.344774 12733 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.227.193
I20260923 23:18:12.346032 12733 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:44325
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790205259943518-25487-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790205259943518-25487-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.227.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790205259943518-25487-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.227.193
--webserver_port=0
--tserver_master_addrs=127.24.227.254:43997
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12733
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.227.193
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790205259943518-25487-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:18:12.346231 12733 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:18:12.346493 12733 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:18:12.351218 12739 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:18:12.351291 12733 server_base.cc:1034] running on GCE node
W20260923 23:18:12.351411 12738 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:18:12.351213 12741 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:18:12.351622 12733 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:18:12.351907 12733 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:18:12.353050 12733 hybrid_clock.cc:648] HybridClock initialized: now 1790205492353035 us; error 35 us; skew 500 ppm
I20260923 23:18:12.354282 12733 webserver.cc:460] Webserver started at http://127.24.227.193:38755/ using document root <none> and password file <none>
I20260923 23:18:12.354516 12733 fs_manager.cc:359] Metadata directory not provided
I20260923 23:18:12.354583 12733 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:18:12.354719 12733 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:18:12.355656 12733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790205259943518-25487-0/minicluster-data/ts-0/data/instance:
uuid: "459e4f2a94f84986ae3b9848fa476b41"
format_stamp: "Formatted at 2026-09-23 23:18:12 on dist-test-slave-754n"
server_key: "132691ddeb0f7543232d2fff1f6e5318"
server_key_iv: "1203618babf1a8c30851c449e8319eb2"
server_key_version: "encryptionkey@0"
I20260923 23:18:12.355999 12733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790205259943518-25487-0/minicluster-data/ts-0/wal/instance:
uuid: "459e4f2a94f84986ae3b9848fa476b41"
format_stamp: "Formatted at 2026-09-23 23:18:12 on dist-test-slave-754n"
server_key: "132691ddeb0f7543232d2fff1f6e5318"
server_key_iv: "1203618babf1a8c30851c449e8319eb2"
server_key_version: "encryptionkey@0"
I20260923 23:18:12.357236 12733 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 23:18:12.357951 12747 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:18:12.358119 12733 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 23:18:12.358203 12733 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790205259943518-25487-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790205259943518-25487-0/minicluster-data/ts-0/wal
uuid: "459e4f2a94f84986ae3b9848fa476b41"
format_stamp: "Formatted at 2026-09-23 23:18:12 on dist-test-slave-754n"
server_key: "132691ddeb0f7543232d2fff1f6e5318"
server_key_iv: "1203618babf1a8c30851c449e8319eb2"
server_key_version: "encryptionkey@0"
I20260923 23:18:12.358286 12733 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790205259943518-25487-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790205259943518-25487-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790205259943518-25487-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:18:12.382434 12733 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:18:12.382699 12733 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:18:12.382850 12733 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:18:12.383050 12733 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 23:18:12.383359 12733 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 23:18:12.383412 12733 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 23:18:12.383452 12733 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 23:18:12.383478 12733 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 23:18:12.388875 12733 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.193:44549
I20260923 23:18:12.388957 12860 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.193:44549 every 8 connection(s)
I20260923 23:18:12.389242 12733 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790205259943518-25487-0/minicluster-data/ts-0/data/info.pb
I20260923 23:18:12.391218 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 12733
I20260923 23:18:12.391342 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790205259943518-25487-0/minicluster-data/ts-0/wal/instance
I20260923 23:18:12.391548 25487 external_mini_cluster.cc:1428] Setting key 390cbbf7c1255f69090705d535447932
I20260923 23:18:12.392510 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790205259943518-25487-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790205259943518-25487-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790205259943518-25487-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790205259943518-25487-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.227.194:0
--local_ip_for_outbound_sockets=127.24.227.194
--webserver_interface=127.24.227.194
--webserver_port=0
--tserver_master_addrs=127.24.227.254:43997
--builtin_ntp_servers=127.24.227.212:44325
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260923 23:18:12.394714 12712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:12.389955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.193:54943 (local address 127.24.227.254:43997)
0923 23:18:12.390025 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:12.390027 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:12.390254 (+ 227us) server_negotiation.cc:407] Connection header received
0923 23:18:12.390665 (+ 411us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:12.390666 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:12.390677 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:12.390731 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:12.391494 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:12.392141 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:12.393203 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:12.393513 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:12.393600 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:12.394466 (+ 866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:18:12.394467 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:18:12.394467 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 23:18:12.394471 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:18:12.394483 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 23:18:12.394487 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:18:12.394525 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:12.394631 (+ 106us) server_negotiation.cc:299] Negotiation successful
0923 23:18:12.394659 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260923 23:18:12.394964 12863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:12.389941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:43997 (local address 127.24.227.193:54943)
0923 23:18:12.390136 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:12.390165 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0923 23:18:12.390585 (+ 420us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:18:12.390775 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:18:12.390777 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:18:12.390785 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:18:12.391396 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:12.391402 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:12.392231 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:12.392233 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:12.393111 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:12.393114 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:12.393641 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:12.393642 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:12.393809 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:12.394314 (+ 505us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 23:18:12.394314 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:18:12.394352 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 23:18:12.394380 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:18:12.394537 (+ 157us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:18:12.394542 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:18:12.394545 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 23:18:12.394626 (+ 81us) client_negotiation.cc:240] Negotiation successful
0923 23:18:12.394727 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":113,"threads_started":1}
I20260923 23:18:12.395579 12861 heartbeater.cc:347] Connected to a master server at 127.24.227.254:43997
I20260923 23:18:12.395673 12861 heartbeater.cc:464] Registering TS with master...
I20260923 23:18:12.395830 12861 heartbeater.cc:511] Master 127.24.227.254:43997 requested a full tablet report, sending...
I20260923 23:18:12.396426 12673 ts_manager.cc:194] Registered new tserver with Master: 459e4f2a94f84986ae3b9848fa476b41 (127.24.227.193:44549)
I20260923 23:18:12.397389 12673 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.227.193:54943
W20260923 23:18:12.474984 12864 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:18:12.475162 12864 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:18:12.475184 12864 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:18:12.476251 12864 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:18:12.476286 12864 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:18:12.476310 12864 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.227.194
I20260923 23:18:12.477506 12864 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:44325
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790205259943518-25487-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790205259943518-25487-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.227.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790205259943518-25487-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.227.194
--webserver_port=0
--tserver_master_addrs=127.24.227.254:43997
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12864
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.227.194
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790205259943518-25487-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:18:12.477669 12864 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:18:12.477888 12864 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:18:12.482673 12870 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:18:12.482681 12869 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:18:12.482682 12872 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:18:12.483052 12864 server_base.cc:1034] running on GCE node
I20260923 23:18:12.483255 12864 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:18:12.483428 12864 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:18:12.484560 12864 hybrid_clock.cc:648] HybridClock initialized: now 1790205492484549 us; error 27 us; skew 500 ppm
I20260923 23:18:12.485666 12864 webserver.cc:460] Webserver started at http://127.24.227.194:35451/ using document root <none> and password file <none>
I20260923 23:18:12.485886 12864 fs_manager.cc:359] Metadata directory not provided
I20260923 23:18:12.485929 12864 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:18:12.486089 12864 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:18:12.487051 12864 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790205259943518-25487-0/minicluster-data/ts-1/data/instance:
uuid: "c9e0915f60864cdf93b260b895d7a7c9"
format_stamp: "Formatted at 2026-09-23 23:18:12 on dist-test-slave-754n"
server_key: "2bae9ddc1bb5106415d0fc5a795d7f8d"
server_key_iv: "2ed34a96eff07b0f9fb9b7e5cb2f33c1"
server_key_version: "encryptionkey@0"
I20260923 23:18:12.487391 12864 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790205259943518-25487-0/minicluster-data/ts-1/wal/instance:
uuid: "c9e0915f60864cdf93b260b895d7a7c9"
format_stamp: "Formatted at 2026-09-23 23:18:12 on dist-test-slave-754n"
server_key: "2bae9ddc1bb5106415d0fc5a795d7f8d"
server_key_iv: "2ed34a96eff07b0f9fb9b7e5cb2f33c1"
server_key_version: "encryptionkey@0"
I20260923 23:18:12.488716 12864 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 23:18:12.489377 12878 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:18:12.489542 12864 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 23:18:12.489634 12864 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790205259943518-25487-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790205259943518-25487-0/minicluster-data/ts-1/wal
uuid: "c9e0915f60864cdf93b260b895d7a7c9"
format_stamp: "Formatted at 2026-09-23 23:18:12 on dist-test-slave-754n"
server_key: "2bae9ddc1bb5106415d0fc5a795d7f8d"
server_key_iv: "2ed34a96eff07b0f9fb9b7e5cb2f33c1"
server_key_version: "encryptionkey@0"
I20260923 23:18:12.489711 12864 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790205259943518-25487-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790205259943518-25487-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790205259943518-25487-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:18:12.515167 12864 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:18:12.515486 12864 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:18:12.515697 12864 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:18:12.515972 12864 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 23:18:12.516301 12864 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 23:18:12.516356 12864 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 23:18:12.516395 12864 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 23:18:12.516422 12864 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 23:18:12.522149 12864 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.194:41107
I20260923 23:18:12.522198 12991 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.194:41107 every 8 connection(s)
I20260923 23:18:12.522500 12864 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790205259943518-25487-0/minicluster-data/ts-1/data/info.pb
I20260923 23:18:12.527201 12712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:12.523176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.194:49089 (local address 127.24.227.254:43997)
0923 23:18:12.523253 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:12.523256 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:12.523450 (+ 194us) server_negotiation.cc:407] Connection header received
0923 23:18:12.523975 (+ 525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:12.523976 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:12.523986 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:12.524036 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:12.524640 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:12.525114 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:12.526007 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:12.526184 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:12.526233 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:12.527018 (+ 785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:18:12.527018 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:18:12.527019 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 23:18:12.527023 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:18:12.527034 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 23:18:12.527037 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:18:12.527073 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:12.527130 (+ 57us) server_negotiation.cc:299] Negotiation successful
0923 23:18:12.527147 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260923 23:18:12.527427 12994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:12.523144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:43997 (local address 127.24.227.194:49089)
0923 23:18:12.523337 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:12.523369 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0923 23:18:12.523831 (+ 462us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:18:12.524098 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:18:12.524101 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:18:12.524109 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:18:12.524575 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:12.524578 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:12.525188 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:12.525189 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:12.525938 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:12.525939 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:12.526255 (+ 316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:12.526256 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:12.526359 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:12.526903 (+ 544us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 23:18:12.526904 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:18:12.526929 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 23:18:12.526950 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:18:12.527074 (+ 124us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:18:12.527076 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:18:12.527078 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 23:18:12.527122 (+ 44us) client_negotiation.cc:240] Negotiation successful
0923 23:18:12.527193 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":100,"threads_started":1}
I20260923 23:18:12.527974 12992 heartbeater.cc:347] Connected to a master server at 127.24.227.254:43997
I20260923 23:18:12.528074 12992 heartbeater.cc:464] Registering TS with master...
I20260923 23:18:12.528244 12992 heartbeater.cc:511] Master 127.24.227.254:43997 requested a full tablet report, sending...
I20260923 23:18:12.528617 12673 ts_manager.cc:194] Registered new tserver with Master: c9e0915f60864cdf93b260b895d7a7c9 (127.24.227.194:41107)
I20260923 23:18:12.528965 12673 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.227.194:49089
I20260923 23:18:12.531733 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 12864
I20260923 23:18:12.531845 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790205259943518-25487-0/minicluster-data/ts-1/wal/instance
I20260923 23:18:12.532052 25487 external_mini_cluster.cc:1428] Setting key 0184b7f6319f3a4e3ffad670537755a7
I20260923 23:18:12.532943 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790205259943518-25487-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790205259943518-25487-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790205259943518-25487-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790205259943518-25487-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.227.195:0
--local_ip_for_outbound_sockets=127.24.227.195
--webserver_interface=127.24.227.195
--webserver_port=0
--tserver_master_addrs=127.24.227.254:43997
--builtin_ntp_servers=127.24.227.212:44325
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260923 23:18:12.609892 12995 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:18:12.610105 12995 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:18:12.610126 12995 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:18:12.611251 12995 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:18:12.611290 12995 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:18:12.611342 12995 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.227.195
I20260923 23:18:12.612677 12995 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:44325
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790205259943518-25487-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790205259943518-25487-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.227.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790205259943518-25487-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.227.195
--webserver_port=0
--tserver_master_addrs=127.24.227.254:43997
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12995
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.227.195
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790205259943518-25487-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:18:12.612898 12995 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:18:12.613168 12995 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:18:12.617777 13001 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:18:12.617774 13000 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:18:12.617796 13003 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:18:12.618029 12995 server_base.cc:1034] running on GCE node
I20260923 23:18:12.618234 12995 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:18:12.618533 12995 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:18:12.619688 12995 hybrid_clock.cc:648] HybridClock initialized: now 1790205492619682 us; error 37 us; skew 500 ppm
I20260923 23:18:12.620935 12995 webserver.cc:460] Webserver started at http://127.24.227.195:35933/ using document root <none> and password file <none>
I20260923 23:18:12.621178 12995 fs_manager.cc:359] Metadata directory not provided
I20260923 23:18:12.621225 12995 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:18:12.621390 12995 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:18:12.622349 12995 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790205259943518-25487-0/minicluster-data/ts-2/data/instance:
uuid: "44028e2708f9488e9234d42c0cfdfa16"
format_stamp: "Formatted at 2026-09-23 23:18:12 on dist-test-slave-754n"
server_key: "453ada02690dee9cd8217745d5717c97"
server_key_iv: "42db3393d536138057ea1d57d13705ab"
server_key_version: "encryptionkey@0"
I20260923 23:18:12.622713 12995 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790205259943518-25487-0/minicluster-data/ts-2/wal/instance:
uuid: "44028e2708f9488e9234d42c0cfdfa16"
format_stamp: "Formatted at 2026-09-23 23:18:12 on dist-test-slave-754n"
server_key: "453ada02690dee9cd8217745d5717c97"
server_key_iv: "42db3393d536138057ea1d57d13705ab"
server_key_version: "encryptionkey@0"
I20260923 23:18:12.623973 12995 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 23:18:12.624656 13009 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:18:12.624826 12995 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 23:18:12.624915 12995 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790205259943518-25487-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790205259943518-25487-0/minicluster-data/ts-2/wal
uuid: "44028e2708f9488e9234d42c0cfdfa16"
format_stamp: "Formatted at 2026-09-23 23:18:12 on dist-test-slave-754n"
server_key: "453ada02690dee9cd8217745d5717c97"
server_key_iv: "42db3393d536138057ea1d57d13705ab"
server_key_version: "encryptionkey@0"
I20260923 23:18:12.624996 12995 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790205259943518-25487-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790205259943518-25487-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790205259943518-25487-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:18:12.650584 12995 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:18:12.650876 12995 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:18:12.651023 12995 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:18:12.651247 12995 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 23:18:12.651557 12995 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 23:18:12.651611 12995 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 23:18:12.651652 12995 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 23:18:12.651679 12995 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 23:18:12.657285 12995 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.195:34271
I20260923 23:18:12.657325 13122 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.195:34271 every 8 connection(s)
I20260923 23:18:12.657639 12995 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790205259943518-25487-0/minicluster-data/ts-2/data/info.pb
I20260923 23:18:12.662667 12712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:12.658276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.195:46233 (local address 127.24.227.254:43997)
0923 23:18:12.658340 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:12.658342 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:12.658545 (+ 203us) server_negotiation.cc:407] Connection header received
0923 23:18:12.659077 (+ 532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:12.659079 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:12.659090 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:12.659164 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:12.659830 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:12.660367 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:12.661351 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:12.661547 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:12.661630 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:12.662378 (+ 748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:18:12.662379 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:18:12.662380 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 23:18:12.662384 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:18:12.662399 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 23:18:12.662403 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:18:12.662457 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:12.662560 (+ 103us) server_negotiation.cc:299] Negotiation successful
0923 23:18:12.662586 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260923 23:18:12.662899 13125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:12.658259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:43997 (local address 127.24.227.195:46233)
0923 23:18:12.658429 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:12.658460 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0923 23:18:12.658968 (+ 508us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:18:12.659229 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:18:12.659233 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:18:12.659245 (+ 12us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:18:12.659731 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:12.659736 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:12.660472 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:12.660473 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:12.661246 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:12.661249 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:12.661641 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:12.661642 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:12.661777 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:12.662242 (+ 465us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 23:18:12.662243 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:18:12.662269 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 23:18:12.662290 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:18:12.662462 (+ 172us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:18:12.662466 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:18:12.662469 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 23:18:12.662550 (+ 81us) client_negotiation.cc:240] Negotiation successful
0923 23:18:12.662659 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":91,"threads_started":1}
I20260923 23:18:12.663389 13123 heartbeater.cc:347] Connected to a master server at 127.24.227.254:43997
I20260923 23:18:12.663481 13123 heartbeater.cc:464] Registering TS with master...
I20260923 23:18:12.663655 13123 heartbeater.cc:511] Master 127.24.227.254:43997 requested a full tablet report, sending...
I20260923 23:18:12.664039 12673 ts_manager.cc:194] Registered new tserver with Master: 44028e2708f9488e9234d42c0cfdfa16 (127.24.227.195:34271)
I20260923 23:18:12.664363 12673 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.227.195:46233
I20260923 23:18:12.667695 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 12995
I20260923 23:18:12.667801 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790205259943518-25487-0/minicluster-data/ts-2/wal/instance
I20260923 23:18:12.668010 25487 external_mini_cluster.cc:1428] Setting key 6f10f0284327c4b6f20b5d6fff5b56bd
I20260923 23:18:12.668809 25487 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 23:18:12.672806 12712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:12.669995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35752 (local address 127.24.227.254:43997)
0923 23:18:12.670075 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:12.670077 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:12.670215 (+ 138us) server_negotiation.cc:407] Connection header received
0923 23:18:12.670288 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:12.670289 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:12.670312 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:12.670360 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:12.670514 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:12.670962 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:12.671710 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:12.671985 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:12.672052 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:12.672585 (+ 533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:18:12.672586 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:18:12.672586 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 23:18:12.672590 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:18:12.672602 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 23:18:12.672606 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:18:12.672656 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:12.672715 (+ 59us) server_negotiation.cc:299] Negotiation successful
0923 23:18:12.672732 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260923 23:18:12.674136 12673 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35752:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 23:18:12.675585 12673 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 23:18:12.681509 13135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:12.677823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.194:41107 (local address 127.0.0.1:32952)
0923 23:18:12.678126 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:12.678135 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 23:18:12.678189 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:18:12.678441 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:18:12.678442 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:18:12.678443 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:18:12.678482 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:12.678483 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:12.679419 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:12.679420 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:12.680565 (+ 1145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:12.680567 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:12.681239 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:12.681241 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:12.681315 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:12.681328 (+ 13us) client_negotiation.cc:769] Sending connection context
0923 23:18:12.681387 (+ 59us) client_negotiation.cc:240] Negotiation successful
0923 23:18:12.681438 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"mutex_wait_us":35,"thread_start_us":73,"threads_started":1}
I20260923 23:18:12.681521 13134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:12.677934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32952 (local address 127.24.227.194:41107)
0923 23:18:12.678120 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:12.678124 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:12.678252 (+ 128us) server_negotiation.cc:407] Connection header received
0923 23:18:12.678298 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:12.678301 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:12.678317 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:12.678385 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:18:12.678557 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:12.679329 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:12.680660 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:12.681131 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:12.681234 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:12.681318 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:12.681409 (+ 91us) server_negotiation.cc:299] Negotiation successful
0923 23:18:12.681448 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":86,"threads_started":1}
I20260923 23:18:12.681938 13132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:12.677745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60270 (local address 127.24.227.193:44549)
0923 23:18:12.677965 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:12.677970 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:12.677981 (+ 11us) server_negotiation.cc:407] Connection header received
0923 23:18:12.678025 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:12.678027 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:12.678042 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:12.678126 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:18:12.678375 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:12.679130 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:12.680626 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:12.681334 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:12.681409 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:12.681483 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:12.681820 (+ 337us) server_negotiation.cc:299] Negotiation successful
0923 23:18:12.681863 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":107,"threads_started":1}
I20260923 23:18:12.682188 13131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:12.677626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.193:44549 (local address 127.0.0.1:60270)
0923 23:18:12.677735 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:12.677771 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0923 23:18:12.677827 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:18:12.678205 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:18:12.678208 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:18:12.678210 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:18:12.678293 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:12.678294 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:12.679213 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:12.679214 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:12.680538 (+ 1324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:12.680541 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:12.681652 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:12.681653 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:12.681716 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:12.681721 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 23:18:12.682076 (+ 355us) client_negotiation.cc:240] Negotiation successful
0923 23:18:12.682120 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88,"mutex_wait_us":4,"thread_start_us":67,"threads_started":1}
I20260923 23:18:12.682199 12926 tablet_service.cc:1467] Processing CreateTablet for tablet 842683a78db3481c9976e82eb1a72e01 (DEFAULT_TABLE table=test-table [id=daedc426bffd404e88f0d99cdc6475f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 23:18:12.682582 12926 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 842683a78db3481c9976e82eb1a72e01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:18:12.682879 13133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:12.677998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.195:34271 (local address 127.0.0.1:60940)
0923 23:18:12.678189 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:12.678199 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 23:18:12.678243 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:18:12.679470 (+ 1227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:18:12.679471 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:18:12.679472 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:18:12.679504 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:12.679505 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:12.680513 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:12.680514 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:12.681662 (+ 1148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:12.681665 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:12.682552 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:12.682553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:12.682612 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:12.682615 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 23:18:12.682767 (+ 152us) client_negotiation.cc:240] Negotiation successful
0923 23:18:12.682811 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"mutex_wait_us":2,"thread_start_us":71,"threads_started":1}
I20260923 23:18:12.682893 13136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:12.678074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60940 (local address 127.24.227.195:34271)
0923 23:18:12.679277 (+ 1203us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:12.679281 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:12.679290 (+ 9us) server_negotiation.cc:407] Connection header received
0923 23:18:12.679322 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:12.679324 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:12.679339 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:12.679410 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:18:12.679586 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:12.680448 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:12.681755 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:12.682446 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:12.682540 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:12.682618 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:12.682783 (+ 165us) server_negotiation.cc:299] Negotiation successful
0923 23:18:12.682820 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1145,"thread_start_us":67,"threads_started":1}
I20260923 23:18:12.684494 13057 tablet_service.cc:1467] Processing CreateTablet for tablet 842683a78db3481c9976e82eb1a72e01 (DEFAULT_TABLE table=test-table [id=daedc426bffd404e88f0d99cdc6475f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 23:18:12.684633 12795 tablet_service.cc:1467] Processing CreateTablet for tablet 842683a78db3481c9976e82eb1a72e01 (DEFAULT_TABLE table=test-table [id=daedc426bffd404e88f0d99cdc6475f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 23:18:12.684818 13057 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 842683a78db3481c9976e82eb1a72e01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:18:12.684955 12795 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 842683a78db3481c9976e82eb1a72e01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:18:12.685740 13137 tablet_bootstrap.cc:492] T 842683a78db3481c9976e82eb1a72e01 P c9e0915f60864cdf93b260b895d7a7c9: Bootstrap starting.
I20260923 23:18:12.686441 13137 tablet_bootstrap.cc:654] T 842683a78db3481c9976e82eb1a72e01 P c9e0915f60864cdf93b260b895d7a7c9: Neither blocks nor log segments found. Creating new log.
I20260923 23:18:12.686784 13137 log.cc:824] T 842683a78db3481c9976e82eb1a72e01 P c9e0915f60864cdf93b260b895d7a7c9: Log is configured to *not* fsync() on all Append() calls
I20260923 23:18:12.687559 13137 tablet_bootstrap.cc:492] T 842683a78db3481c9976e82eb1a72e01 P c9e0915f60864cdf93b260b895d7a7c9: No bootstrap required, opened a new log
I20260923 23:18:12.687631 13137 ts_tablet_manager.cc:1397] T 842683a78db3481c9976e82eb1a72e01 P c9e0915f60864cdf93b260b895d7a7c9: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 23:18:12.687834 13139 tablet_bootstrap.cc:492] T 842683a78db3481c9976e82eb1a72e01 P 459e4f2a94f84986ae3b9848fa476b41: Bootstrap starting.
I20260923 23:18:12.688482 13139 tablet_bootstrap.cc:654] T 842683a78db3481c9976e82eb1a72e01 P 459e4f2a94f84986ae3b9848fa476b41: Neither blocks nor log segments found. Creating new log.
I20260923 23:18:12.688766 13139 log.cc:824] T 842683a78db3481c9976e82eb1a72e01 P 459e4f2a94f84986ae3b9848fa476b41: Log is configured to *not* fsync() on all Append() calls
I20260923 23:18:12.689394 13139 tablet_bootstrap.cc:492] T 842683a78db3481c9976e82eb1a72e01 P 459e4f2a94f84986ae3b9848fa476b41: No bootstrap required, opened a new log
I20260923 23:18:12.689471 13139 ts_tablet_manager.cc:1397] T 842683a78db3481c9976e82eb1a72e01 P 459e4f2a94f84986ae3b9848fa476b41: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 23:18:12.689772 13137 raft_consensus.cc:348] T 842683a78db3481c9976e82eb1a72e01 P c9e0915f60864cdf93b260b895d7a7c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "459e4f2a94f84986ae3b9848fa476b41" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 44549 } } peers { permanent_uuid: "c9e0915f60864cdf93b260b895d7a7c9" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 41107 } } peers { permanent_uuid: "44028e2708f9488e9234d42c0cfdfa16" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 34271 } }
I20260923 23:18:12.689913 13137 raft_consensus.cc:374] T 842683a78db3481c9976e82eb1a72e01 P c9e0915f60864cdf93b260b895d7a7c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:18:12.689934 13137 raft_consensus.cc:729] T 842683a78db3481c9976e82eb1a72e01 P c9e0915f60864cdf93b260b895d7a7c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9e0915f60864cdf93b260b895d7a7c9, State: Initialized, Role: FOLLOWER
I20260923 23:18:12.690081 13137 consensus_queue.cc:260] T 842683a78db3481c9976e82eb1a72e01 P c9e0915f60864cdf93b260b895d7a7c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "459e4f2a94f84986ae3b9848fa476b41" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 44549 } } peers { permanent_uuid: "c9e0915f60864cdf93b260b895d7a7c9" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 41107 } } peers { permanent_uuid: "44028e2708f9488e9234d42c0cfdfa16" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 34271 } }
I20260923 23:18:12.690335 13137 ts_tablet_manager.cc:1428] T 842683a78db3481c9976e82eb1a72e01 P c9e0915f60864cdf93b260b895d7a7c9: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 23:18:12.690382 12992 heartbeater.cc:503] Master 127.24.227.254:43997 was elected leader, sending a full tablet report...
I20260923 23:18:12.690532 13140 tablet_bootstrap.cc:492] T 842683a78db3481c9976e82eb1a72e01 P 44028e2708f9488e9234d42c0cfdfa16: Bootstrap starting.
I20260923 23:18:12.691152 13140 tablet_bootstrap.cc:654] T 842683a78db3481c9976e82eb1a72e01 P 44028e2708f9488e9234d42c0cfdfa16: Neither blocks nor log segments found. Creating new log.
I20260923 23:18:12.691464 13140 log.cc:824] T 842683a78db3481c9976e82eb1a72e01 P 44028e2708f9488e9234d42c0cfdfa16: Log is configured to *not* fsync() on all Append() calls
I20260923 23:18:12.692117 13140 tablet_bootstrap.cc:492] T 842683a78db3481c9976e82eb1a72e01 P 44028e2708f9488e9234d42c0cfdfa16: No bootstrap required, opened a new log
I20260923 23:18:12.692169 13140 ts_tablet_manager.cc:1397] T 842683a78db3481c9976e82eb1a72e01 P 44028e2708f9488e9234d42c0cfdfa16: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 23:18:12.692099 13139 raft_consensus.cc:348] T 842683a78db3481c9976e82eb1a72e01 P 459e4f2a94f84986ae3b9848fa476b41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "459e4f2a94f84986ae3b9848fa476b41" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 44549 } } peers { permanent_uuid: "c9e0915f60864cdf93b260b895d7a7c9" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 41107 } } peers { permanent_uuid: "44028e2708f9488e9234d42c0cfdfa16" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 34271 } }
I20260923 23:18:12.692253 13139 raft_consensus.cc:374] T 842683a78db3481c9976e82eb1a72e01 P 459e4f2a94f84986ae3b9848fa476b41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:18:12.692298 13139 raft_consensus.cc:729] T 842683a78db3481c9976e82eb1a72e01 P 459e4f2a94f84986ae3b9848fa476b41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 459e4f2a94f84986ae3b9848fa476b41, State: Initialized, Role: FOLLOWER
I20260923 23:18:12.692404 13139 consensus_queue.cc:260] T 842683a78db3481c9976e82eb1a72e01 P 459e4f2a94f84986ae3b9848fa476b41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "459e4f2a94f84986ae3b9848fa476b41" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 44549 } } peers { permanent_uuid: "c9e0915f60864cdf93b260b895d7a7c9" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 41107 } } peers { permanent_uuid: "44028e2708f9488e9234d42c0cfdfa16" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 34271 } }
I20260923 23:18:12.692646 12861 heartbeater.cc:503] Master 127.24.227.254:43997 was elected leader, sending a full tablet report...
I20260923 23:18:12.692844 13139 ts_tablet_manager.cc:1428] T 842683a78db3481c9976e82eb1a72e01 P 459e4f2a94f84986ae3b9848fa476b41: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 23:18:12.694376 13140 raft_consensus.cc:348] T 842683a78db3481c9976e82eb1a72e01 P 44028e2708f9488e9234d42c0cfdfa16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "459e4f2a94f84986ae3b9848fa476b41" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 44549 } } peers { permanent_uuid: "c9e0915f60864cdf93b260b895d7a7c9" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 41107 } } peers { permanent_uuid: "44028e2708f9488e9234d42c0cfdfa16" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 34271 } }
I20260923 23:18:12.694492 13140 raft_consensus.cc:374] T 842683a78db3481c9976e82eb1a72e01 P 44028e2708f9488e9234d42c0cfdfa16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:18:12.694514 13140 raft_consensus.cc:729] T 842683a78db3481c9976e82eb1a72e01 P 44028e2708f9488e9234d42c0cfdfa16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44028e2708f9488e9234d42c0cfdfa16, State: Initialized, Role: FOLLOWER
I20260923 23:18:12.694612 13140 consensus_queue.cc:260] T 842683a78db3481c9976e82eb1a72e01 P 44028e2708f9488e9234d42c0cfdfa16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "459e4f2a94f84986ae3b9848fa476b41" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 44549 } } peers { permanent_uuid: "c9e0915f60864cdf93b260b895d7a7c9" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 41107 } } peers { permanent_uuid: "44028e2708f9488e9234d42c0cfdfa16" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 34271 } }
I20260923 23:18:12.694885 13123 heartbeater.cc:503] Master 127.24.227.254:43997 was elected leader, sending a full tablet report...
I20260923 23:18:12.695070 13140 ts_tablet_manager.cc:1428] T 842683a78db3481c9976e82eb1a72e01 P 44028e2708f9488e9234d42c0cfdfa16: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 23:18:12.757157 13145 raft_consensus.cc:482] T 842683a78db3481c9976e82eb1a72e01 P 44028e2708f9488e9234d42c0cfdfa16 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 23:18:12.757300 13145 raft_consensus.cc:504] T 842683a78db3481c9976e82eb1a72e01 P 44028e2708f9488e9234d42c0cfdfa16 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "459e4f2a94f84986ae3b9848fa476b41" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 44549 } } peers { permanent_uuid: "c9e0915f60864cdf93b260b895d7a7c9" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 41107 } } peers { permanent_uuid: "44028e2708f9488e9234d42c0cfdfa16" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 34271 } }
I20260923 23:18:12.757653 13145 leader_election.cc:288] T 842683a78db3481c9976e82eb1a72e01 P 44028e2708f9488e9234d42c0cfdfa16 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 459e4f2a94f84986ae3b9848fa476b41 (127.24.227.193:44549), c9e0915f60864cdf93b260b895d7a7c9 (127.24.227.194:41107)
I20260923 23:18:12.760579 13125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:12.757817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.193:44549 (local address 127.24.227.195:34037)
0923 23:18:12.757874 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:12.757886 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 23:18:12.757950 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:18:12.758204 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:18:12.758205 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:18:12.758208 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:18:12.758288 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:12.758290 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:12.758961 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:12.758962 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:12.759760 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:12.759762 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:12.760325 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:12.760326 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:12.760397 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:12.760400 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 23:18:12.760452 (+ 52us) client_negotiation.cc:240] Negotiation successful
0923 23:18:12.760486 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260923 23:18:12.760615 13132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:12.757898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.195:34037 (local address 127.24.227.193:44549)
0923 23:18:12.757945 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:12.757947 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:12.757958 (+ 11us) server_negotiation.cc:407] Connection header received
0923 23:18:12.758058 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:12.758060 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:12.758075 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:12.758138 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:18:12.758364 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:12.758885 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:12.759849 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:12.760252 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:12.760307 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:12.760313 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:12.760475 (+ 162us) server_negotiation.cc:299] Negotiation successful
0923 23:18:12.760524 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260923 23:18:12.760664 13134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:12.758019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.227.195:55443 (local address 127.24.227.194:41107)
0923 23:18:12.758076 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:12.758078 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:12.758211 (+ 133us) server_negotiation.cc:407] Connection header received
0923 23:18:12.758240 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:12.758241 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:12.758254 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:12.758316 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 23:18:12.758486 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:12.758954 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:12.759922 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:12.760277 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:12.760367 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:12.760374 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:12.760548 (+ 174us) server_negotiation.cc:299] Negotiation successful
0923 23:18:12.760590 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260923 23:18:12.760735 13146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:12.757923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.194:41107 (local address 127.24.227.195:55443)
0923 23:18:12.758091 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:12.758101 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 23:18:12.758154 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:18:12.758339 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:18:12.758340 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:18:12.758342 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 23:18:12.758412 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:12.758414 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:12.759029 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:12.759030 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:12.759775 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:12.759777 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:12.760391 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:12.760392 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:12.760461 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:12.760465 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 23:18:12.760521 (+ 56us) client_negotiation.cc:240] Negotiation successful
0923 23:18:12.760566 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":101,"threads_started":1}
I20260923 23:18:12.760954 12815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "842683a78db3481c9976e82eb1a72e01" candidate_uuid: "44028e2708f9488e9234d42c0cfdfa16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "459e4f2a94f84986ae3b9848fa476b41" is_pre_election: true
I20260923 23:18:12.761126 12815 raft_consensus.cc:2454] T 842683a78db3481c9976e82eb1a72e01 P 459e4f2a94f84986ae3b9848fa476b41 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 44028e2708f9488e9234d42c0cfdfa16 in term 0.
I20260923 23:18:12.761219 12946 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "842683a78db3481c9976e82eb1a72e01" candidate_uuid: "44028e2708f9488e9234d42c0cfdfa16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9e0915f60864cdf93b260b895d7a7c9" is_pre_election: true
I20260923 23:18:12.761368 12946 raft_consensus.cc:2454] T 842683a78db3481c9976e82eb1a72e01 P c9e0915f60864cdf93b260b895d7a7c9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 44028e2708f9488e9234d42c0cfdfa16 in term 0.
I20260923 23:18:12.761397 13013 leader_election.cc:302] T 842683a78db3481c9976e82eb1a72e01 P 44028e2708f9488e9234d42c0cfdfa16 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44028e2708f9488e9234d42c0cfdfa16, 459e4f2a94f84986ae3b9848fa476b41; no voters:
I20260923 23:18:12.761520 13145 raft_consensus.cc:2781] T 842683a78db3481c9976e82eb1a72e01 P 44028e2708f9488e9234d42c0cfdfa16 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 23:18:12.761602 13145 raft_consensus.cc:482] T 842683a78db3481c9976e82eb1a72e01 P 44028e2708f9488e9234d42c0cfdfa16 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 23:18:12.761631 13145 raft_consensus.cc:3037] T 842683a78db3481c9976e82eb1a72e01 P 44028e2708f9488e9234d42c0cfdfa16 [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:18:12.762619 13145 raft_consensus.cc:504] T 842683a78db3481c9976e82eb1a72e01 P 44028e2708f9488e9234d42c0cfdfa16 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "459e4f2a94f84986ae3b9848fa476b41" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 44549 } } peers { permanent_uuid: "c9e0915f60864cdf93b260b895d7a7c9" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 41107 } } peers { permanent_uuid: "44028e2708f9488e9234d42c0cfdfa16" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 34271 } }
I20260923 23:18:12.762722 13145 leader_election.cc:288] T 842683a78db3481c9976e82eb1a72e01 P 44028e2708f9488e9234d42c0cfdfa16 [CANDIDATE]: Term 1 election: Requested vote from peers 459e4f2a94f84986ae3b9848fa476b41 (127.24.227.193:44549), c9e0915f60864cdf93b260b895d7a7c9 (127.24.227.194:41107)
I20260923 23:18:12.762923 12815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "842683a78db3481c9976e82eb1a72e01" candidate_uuid: "44028e2708f9488e9234d42c0cfdfa16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "459e4f2a94f84986ae3b9848fa476b41"
I20260923 23:18:12.762977 12946 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "842683a78db3481c9976e82eb1a72e01" candidate_uuid: "44028e2708f9488e9234d42c0cfdfa16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9e0915f60864cdf93b260b895d7a7c9"
I20260923 23:18:12.763032 12946 raft_consensus.cc:3037] T 842683a78db3481c9976e82eb1a72e01 P c9e0915f60864cdf93b260b895d7a7c9 [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:18:12.763036 12815 raft_consensus.cc:3037] T 842683a78db3481c9976e82eb1a72e01 P 459e4f2a94f84986ae3b9848fa476b41 [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:18:12.763759 12946 raft_consensus.cc:2454] T 842683a78db3481c9976e82eb1a72e01 P c9e0915f60864cdf93b260b895d7a7c9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 44028e2708f9488e9234d42c0cfdfa16 in term 1.
I20260923 23:18:12.763782 12815 raft_consensus.cc:2454] T 842683a78db3481c9976e82eb1a72e01 P 459e4f2a94f84986ae3b9848fa476b41 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 44028e2708f9488e9234d42c0cfdfa16 in term 1.
I20260923 23:18:12.763960 13013 leader_election.cc:302] T 842683a78db3481c9976e82eb1a72e01 P 44028e2708f9488e9234d42c0cfdfa16 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44028e2708f9488e9234d42c0cfdfa16, 459e4f2a94f84986ae3b9848fa476b41; no voters:
I20260923 23:18:12.764060 13145 raft_consensus.cc:2781] T 842683a78db3481c9976e82eb1a72e01 P 44028e2708f9488e9234d42c0cfdfa16 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 23:18:12.764186 13145 raft_consensus.cc:686] T 842683a78db3481c9976e82eb1a72e01 P 44028e2708f9488e9234d42c0cfdfa16 [term 1 LEADER]: Becoming Leader. State: Replica: 44028e2708f9488e9234d42c0cfdfa16, State: Running, Role: LEADER
I20260923 23:18:12.764289 13145 consensus_queue.cc:237] T 842683a78db3481c9976e82eb1a72e01 P 44028e2708f9488e9234d42c0cfdfa16 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "459e4f2a94f84986ae3b9848fa476b41" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 44549 } } peers { permanent_uuid: "c9e0915f60864cdf93b260b895d7a7c9" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 41107 } } peers { permanent_uuid: "44028e2708f9488e9234d42c0cfdfa16" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 34271 } }
I20260923 23:18:12.765143 12672 catalog_manager.cc:5495] T 842683a78db3481c9976e82eb1a72e01 P 44028e2708f9488e9234d42c0cfdfa16 reported cstate change: term changed from 0 to 1, leader changed from <none> to 44028e2708f9488e9234d42c0cfdfa16 (127.24.227.195). New cstate: current_term: 1 leader_uuid: "44028e2708f9488e9234d42c0cfdfa16" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "459e4f2a94f84986ae3b9848fa476b41" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 44549 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c9e0915f60864cdf93b260b895d7a7c9" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 41107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "44028e2708f9488e9234d42c0cfdfa16" member_type: VOTER last_known_addr { host: "127.24.227.195" port: 34271 } health_report { overall_health: HEALTHY } } }
W20260923 23:18:12.772949 12993 tablet.cc:2369] T 842683a78db3481c9976e82eb1a72e01 P c9e0915f60864cdf93b260b895d7a7c9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 23:18:12.774667 13136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:12.772562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60942 (local address 127.24.227.195:34271)
0923 23:18:12.772606 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:12.772608 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:12.772628 (+ 20us) server_negotiation.cc:407] Connection header received
0923 23:18:12.772688 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:12.772688 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:12.772703 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:12.772769 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 23:18:12.772842 (+ 73us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:12.773368 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:12.774081 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:12.774255 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:12.774309 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:12.774417 (+ 108us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 23:18:12.774505 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 23:18:12.774558 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:12.774592 (+ 34us) server_negotiation.cc:299] Negotiation successful
0923 23:18:12.774605 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260923 23:18:12.776438 12815 raft_consensus.cc:1261] T 842683a78db3481c9976e82eb1a72e01 P 459e4f2a94f84986ae3b9848fa476b41 [term 1 FOLLOWER]: Refusing update from remote peer 44028e2708f9488e9234d42c0cfdfa16: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 23:18:12.776437 12946 raft_consensus.cc:1261] T 842683a78db3481c9976e82eb1a72e01 P c9e0915f60864cdf93b260b895d7a7c9 [term 1 FOLLOWER]: Refusing update from remote peer 44028e2708f9488e9234d42c0cfdfa16: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 23:18:12.776772 13145 consensus_queue.cc:1035] T 842683a78db3481c9976e82eb1a72e01 P 44028e2708f9488e9234d42c0cfdfa16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "459e4f2a94f84986ae3b9848fa476b41" member_type: VOTER last_known_addr { host: "127.24.227.193" port: 44549 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 23:18:12.776899 13147 consensus_queue.cc:1035] T 842683a78db3481c9976e82eb1a72e01 P 44028e2708f9488e9234d42c0cfdfa16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c9e0915f60864cdf93b260b895d7a7c9" member_type: VOTER last_known_addr { host: "127.24.227.194" port: 41107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 23:18:12.779248 13151 mvcc.cc:205] Tried to move back new op lower bound from 7332681698409463808 to 7332681698363322368. Current Snapshot: MvccSnapshot[applied={T|T < 7332681698409463808}]
I20260923 23:18:12.781692 12672 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35752:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 23:18:12.781788 12672 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35752:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 23:18:12.782526 12672 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 030bb48fd6104199a923a51c7921cc37: Sending DeleteTablet for 3 replicas of tablet 842683a78db3481c9976e82eb1a72e01
I20260923 23:18:12.782904 12795 tablet_service.cc:1514] Processing DeleteTablet for tablet 842683a78db3481c9976e82eb1a72e01 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 23:18:12 UTC) from {username='slave'} at 127.0.0.1:60270
I20260923 23:18:12.782910 13057 tablet_service.cc:1514] Processing DeleteTablet for tablet 842683a78db3481c9976e82eb1a72e01 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 23:18:12 UTC) from {username='slave'} at 127.0.0.1:60940
I20260923 23:18:12.783103 12926 tablet_service.cc:1514] Processing DeleteTablet for tablet 842683a78db3481c9976e82eb1a72e01 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 23:18:12 UTC) from {username='slave'} at 127.0.0.1:32952
I20260923 23:18:12.783129 13161 tablet_replica.cc:330] T 842683a78db3481c9976e82eb1a72e01 P 44028e2708f9488e9234d42c0cfdfa16: stopping tablet replica
I20260923 23:18:12.783221 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 12733
I20260923 23:18:12.784036 13161 raft_consensus.cc:2229] T 842683a78db3481c9976e82eb1a72e01 P 44028e2708f9488e9234d42c0cfdfa16 [term 1 LEADER]: Raft consensus shutting down.
I20260923 23:18:12.784221 13161 raft_consensus.cc:2258] T 842683a78db3481c9976e82eb1a72e01 P 44028e2708f9488e9234d42c0cfdfa16 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 23:18:12.784636 13161 ts_tablet_manager.cc:1905] T 842683a78db3481c9976e82eb1a72e01 P 44028e2708f9488e9234d42c0cfdfa16: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 23:18:12.786291 13161 ts_tablet_manager.cc:1918] T 842683a78db3481c9976e82eb1a72e01 P 44028e2708f9488e9234d42c0cfdfa16: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 23:18:12.786363 13161 log.cc:1196] T 842683a78db3481c9976e82eb1a72e01 P 44028e2708f9488e9234d42c0cfdfa16: Deleting WAL directory at /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790205259943518-25487-0/minicluster-data/ts-2/wal/wals/842683a78db3481c9976e82eb1a72e01
I20260923 23:18:12.786612 13161 ts_tablet_manager.cc:1939] T 842683a78db3481c9976e82eb1a72e01 P 44028e2708f9488e9234d42c0cfdfa16: Deleting consensus metadata
I20260923 23:18:12.787004 12660 catalog_manager.cc:4841] TS 44028e2708f9488e9234d42c0cfdfa16 (127.24.227.195:34271): tablet 842683a78db3481c9976e82eb1a72e01 (table test-table [id=daedc426bffd404e88f0d99cdc6475f7]) successfully deleted
W20260923 23:18:12.788172 12660 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.227.193:44549 (error 108)
I20260923 23:18:12.788615 13135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:12.788478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.193:44549 (local address 127.0.0.1:60276)
0923 23:18:12.788541 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:12.788580 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.193:44549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260923 23:18:12.788707 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 12864
W20260923 23:18:12.792471 12660 catalog_manager.cc:4584] TS 459e4f2a94f84986ae3b9848fa476b41 (127.24.227.193:44549): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 842683a78db3481c9976e82eb1a72e01: Network error: Client connection negotiation failed: client connection to 127.24.227.193:44549: connect: Connection refused (error 111)
I20260923 23:18:12.795588 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 12995
I20260923 23:18:12.795639 13135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:12.795508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.194:41107 (local address 127.0.0.1:32956)
0923 23:18:12.795575 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:12.795606 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.194:41107: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260923 23:18:12.800328 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 12642
2026-09-23T23:18:12Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (702 ms)
[----------] 6 tests from EncryptionPolicyTest (8248 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790205259943518-25487-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 23:18:12 dist-test-slave-754n krb5kdc[13166](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 23:18:12 dist-test-slave-754n krb5kdc[13166](info): set up 2 sockets
Sep 23 23:18:12 dist-test-slave-754n krb5kdc[13166](info): commencing operation
krb5kdc: starting...
W20260923 23:18:13.807546 25487 mini_kdc.cc:121] Time spent starting KDC: real 0.988s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790205259943518-25487-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790205259943518-25487-0/krb5kdc/test-user.keytab.
Sep 23 23:18:13 dist-test-slave-754n krb5kdc[13166](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205493, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T23:18:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T23:18:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790205259943518-25487-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790205259943518-25487-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 23:18:13.914697 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790205259943518-25487-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790205259943518-25487-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790205259943518-25487-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36535
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790205259943518-25487-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36535
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:18:13.990976 13179 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:18:13.991204 13179 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:18:13.991248 13179 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:18:13.992391 13179 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 23:18:13.992460 13179 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:18:13.992484 13179 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:18:13.992512 13179 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 23:18:13.992544 13179 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 23:18:13.993963 13179 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790205259943518-25487-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790205259943518-25487-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36535
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790205259943518-25487-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36535
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13179
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790205259943518-25487-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:18:13.994220 13179 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:18:13.994436 13179 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:18:13.998893 13187 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:18:13.998935 13184 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:18:13.998940 13185 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:18:13.999095 13179 server_base.cc:1034] running on GCE node
I20260923 23:18:13.999670 13179 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:18:13.999996 13179 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:18:14.001132 13179 hybrid_clock.cc:648] HybridClock initialized: now 1790205494001112 us; error 37 us; skew 500 ppm
Sep 23 23:18:13 dist-test-slave-754n krb5kdc[13166](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205493, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:18:14.003436 13179 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 23:18:14.003922 13179 webserver.cc:460] Webserver started at http://127.0.0.1:37211/ using document root <none> and password file <none>
I20260923 23:18:14.004164 13179 fs_manager.cc:359] Metadata directory not provided
I20260923 23:18:14.004228 13179 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:18:14.004359 13179 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:18:14.005342 13179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790205259943518-25487-0/minicluster-data/master-0/data/instance:
uuid: "480fb033ba1c48beb987d4896932672b"
format_stamp: "Formatted at 2026-09-23 23:18:14 on dist-test-slave-754n"
server_key: "c7e21c6bb97eed80534957d29f4a26ef"
server_key_iv: "768f41b7dc40be196bbba185e955d9c5"
server_key_version: "encryptionkey@0"
I20260923 23:18:14.005699 13179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790205259943518-25487-0/minicluster-data/master-0/wal/instance:
uuid: "480fb033ba1c48beb987d4896932672b"
format_stamp: "Formatted at 2026-09-23 23:18:14 on dist-test-slave-754n"
server_key: "c7e21c6bb97eed80534957d29f4a26ef"
server_key_iv: "768f41b7dc40be196bbba185e955d9c5"
server_key_version: "encryptionkey@0"
I20260923 23:18:14.006985 13179 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 23:18:14.007665 13194 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:18:14.007838 13179 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 23:18:14.007979 13179 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790205259943518-25487-0/minicluster-data/master-0/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790205259943518-25487-0/minicluster-data/master-0/wal
uuid: "480fb033ba1c48beb987d4896932672b"
format_stamp: "Formatted at 2026-09-23 23:18:14 on dist-test-slave-754n"
server_key: "c7e21c6bb97eed80534957d29f4a26ef"
server_key_iv: "768f41b7dc40be196bbba185e955d9c5"
server_key_version: "encryptionkey@0"
I20260923 23:18:14.008065 13179 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790205259943518-25487-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790205259943518-25487-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790205259943518-25487-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:18:14.023329 13179 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:18:14.025125 13179 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:18:14.025226 13179 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:18:14.029027 13179 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36535
I20260923 23:18:14.029075 13246 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36535 every 8 connection(s)
I20260923 23:18:14.029381 13179 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
I20260923 23:18:14.030234 13247 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:18:14.032521 13247 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 480fb033ba1c48beb987d4896932672b: Bootstrap starting.
I20260923 23:18:14.033079 13247 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 480fb033ba1c48beb987d4896932672b: Neither blocks nor log segments found. Creating new log.
I20260923 23:18:14.033356 13247 log.cc:824] T 00000000000000000000000000000000 P 480fb033ba1c48beb987d4896932672b: Log is configured to *not* fsync() on all Append() calls
I20260923 23:18:14.034099 13247 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 480fb033ba1c48beb987d4896932672b: No bootstrap required, opened a new log
I20260923 23:18:14.035699 13247 raft_consensus.cc:348] T 00000000000000000000000000000000 P 480fb033ba1c48beb987d4896932672b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "480fb033ba1c48beb987d4896932672b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36535 } }
I20260923 23:18:14.035868 13247 raft_consensus.cc:374] T 00000000000000000000000000000000 P 480fb033ba1c48beb987d4896932672b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:18:14.035970 13247 raft_consensus.cc:729] T 00000000000000000000000000000000 P 480fb033ba1c48beb987d4896932672b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 480fb033ba1c48beb987d4896932672b, State: Initialized, Role: FOLLOWER
I20260923 23:18:14.036139 13247 consensus_queue.cc:260] T 00000000000000000000000000000000 P 480fb033ba1c48beb987d4896932672b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "480fb033ba1c48beb987d4896932672b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36535 } }
I20260923 23:18:14.036221 13247 raft_consensus.cc:388] T 00000000000000000000000000000000 P 480fb033ba1c48beb987d4896932672b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 23:18:14.036265 13247 raft_consensus.cc:482] T 00000000000000000000000000000000 P 480fb033ba1c48beb987d4896932672b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 23:18:14.036314 13247 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 480fb033ba1c48beb987d4896932672b [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:18:14.036810 13247 raft_consensus.cc:504] T 00000000000000000000000000000000 P 480fb033ba1c48beb987d4896932672b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "480fb033ba1c48beb987d4896932672b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36535 } }
I20260923 23:18:14.036926 13247 leader_election.cc:302] T 00000000000000000000000000000000 P 480fb033ba1c48beb987d4896932672b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 480fb033ba1c48beb987d4896932672b; no voters:
I20260923 23:18:14.037125 13247 leader_election.cc:288] T 00000000000000000000000000000000 P 480fb033ba1c48beb987d4896932672b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 23:18:14.037148 13250 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 480fb033ba1c48beb987d4896932672b [term 1 FOLLOWER]: Leader election won for term 1
I20260923 23:18:14.037317 13250 raft_consensus.cc:686] T 00000000000000000000000000000000 P 480fb033ba1c48beb987d4896932672b [term 1 LEADER]: Becoming Leader. State: Replica: 480fb033ba1c48beb987d4896932672b, State: Running, Role: LEADER
I20260923 23:18:14.037370 13247 sys_catalog.cc:564] T 00000000000000000000000000000000 P 480fb033ba1c48beb987d4896932672b [sys.catalog]: configured and running, proceeding with master startup.
I20260923 23:18:14.037401 13250 consensus_queue.cc:237] T 00000000000000000000000000000000 P 480fb033ba1c48beb987d4896932672b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "480fb033ba1c48beb987d4896932672b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36535 } }
I20260923 23:18:14.037796 13251 sys_catalog.cc:455] T 00000000000000000000000000000000 P 480fb033ba1c48beb987d4896932672b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "480fb033ba1c48beb987d4896932672b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "480fb033ba1c48beb987d4896932672b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36535 } } }
I20260923 23:18:14.037879 13251 sys_catalog.cc:458] T 00000000000000000000000000000000 P 480fb033ba1c48beb987d4896932672b [sys.catalog]: This master's current role is: LEADER
I20260923 23:18:14.038025 13252 sys_catalog.cc:455] T 00000000000000000000000000000000 P 480fb033ba1c48beb987d4896932672b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 480fb033ba1c48beb987d4896932672b. Latest consensus state: current_term: 1 leader_uuid: "480fb033ba1c48beb987d4896932672b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "480fb033ba1c48beb987d4896932672b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36535 } } }
I20260923 23:18:14.038161 13252 sys_catalog.cc:458] T 00000000000000000000000000000000 P 480fb033ba1c48beb987d4896932672b [sys.catalog]: This master's current role is: LEADER
I20260923 23:18:14.038287 13258 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 23:18:14.038689 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 13179
I20260923 23:18:14.038779 13258 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 23:18:14.038794 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790205259943518-25487-0/minicluster-data/master-0/wal/instance
I20260923 23:18:14.038998 25487 external_mini_cluster.cc:1428] Setting key edc836419354c7aa79637df8b5600cc5
I20260923 23:18:14.044757 13258 catalog_manager.cc:1347] Generated new cluster ID: 0fa608e518e8470aba8b9973271dd95c
I20260923 23:18:14.044826 13258 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 23:18:14 dist-test-slave-754n krb5kdc[13166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205493, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260923 23:18:14.049590 13269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:14.039563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45279 (local address 127.0.0.1:36535)
0923 23:18:14.039745 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:14.039755 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:14.039785 (+ 30us) server_negotiation.cc:407] Connection header received
0923 23:18:14.039935 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:14.039942 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:14.040000 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:14.040109 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:14.040849 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:14.041708 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:14.042778 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:14.043278 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:14.043357 (+ 79us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:14.046015 (+ 2658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:18:14.046019 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:18:14.046019 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:18:14.046029 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:18:14.047852 (+ 1823us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:14.048320 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:14.048323 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:14.048323 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:14.048442 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:14.048688 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:14.048689 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:14.048689 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:14.048998 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:18:14.049159 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:14.049261 (+ 102us) server_negotiation.cc:299] Negotiation successful
0923 23:18:14.049378 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":100,"threads_started":1}
I20260923 23:18:14.050577 25487 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 23:18:14.058691 13269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:14.052029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42293 (local address 127.0.0.1:36535)
0923 23:18:14.052062 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:14.052064 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:14.052183 (+ 119us) server_negotiation.cc:407] Connection header received
0923 23:18:14.052209 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:14.052209 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:14.052218 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:14.052286 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:14.052818 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:14.053566 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:14.054493 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:14.054829 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:14.054905 (+ 76us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:14.056131 (+ 1226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:18:14.056133 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:18:14.056133 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:18:14.056139 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:18:14.057631 (+ 1492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:14.057988 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:14.057989 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:14.057990 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:14.058044 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:14.058251 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:14.058251 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:14.058252 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:14.058364 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:18:14.058455 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:14.058557 (+ 102us) server_negotiation.cc:299] Negotiation successful
0923 23:18:14.058612 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260923 23:18:14.066169 13258 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 23:18:14.066867 13258 catalog_manager.cc:1504] Loading token signing keys...
I20260923 23:18:14.076769 13258 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 480fb033ba1c48beb987d4896932672b: Generated new TSK 0
I20260923 23:18:14.076929 13258 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 23:18:14.086153 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 13179
W20260923 23:18:14.090250 13164 connection.cc:535] client connection to 127.0.0.1:36535 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36535): Connection reset by peer (error 104)
2026-09-23T23:18:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1284 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790205259943518-25487-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 23:18:14 dist-test-slave-754n krb5kdc[13278](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 23:18:14 dist-test-slave-754n krb5kdc[13278](info): set up 2 sockets
Sep 23 23:18:14 dist-test-slave-754n krb5kdc[13278](info): commencing operation
krb5kdc: starting...
W20260923 23:18:15.079692 25487 mini_kdc.cc:121] Time spent starting KDC: real 0.976s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790205259943518-25487-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790205259943518-25487-0/krb5kdc/test-user.keytab.
Sep 23 23:18:15 dist-test-slave-754n krb5kdc[13278](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205495, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T23:18:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T23:18:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790205259943518-25487-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790205259943518-25487-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 23:18:15.187311 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790205259943518-25487-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790205259943518-25487-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790205259943518-25487-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34377
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790205259943518-25487-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34377
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:18:15.264030 13291 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:18:15.264227 13291 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:18:15.264263 13291 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:18:15.265357 13291 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 23:18:15.265421 13291 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:18:15.265453 13291 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:18:15.265501 13291 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 23:18:15.265529 13291 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 23:18:15.266888 13291 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790205259943518-25487-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790205259943518-25487-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34377
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790205259943518-25487-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34377
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13291
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790205259943518-25487-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:18:15.267143 13291 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:18:15.267349 13291 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:18:15.271764 13299 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:18:15.272013 13297 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:18:15.272037 13291 server_base.cc:1034] running on GCE node
W20260923 23:18:15.271754 13296 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:18:15.272378 13291 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:18:15.272681 13291 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:18:15.273839 13291 hybrid_clock.cc:648] HybridClock initialized: now 1790205495273823 us; error 47 us; skew 500 ppm
Sep 23 23:18:15 dist-test-slave-754n krb5kdc[13278](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205495, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:18:15.281185 13291 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 23:18:15.281636 13291 webserver.cc:460] Webserver started at http://127.0.0.1:36567/ using document root <none> and password file <none>
I20260923 23:18:15.281862 13291 fs_manager.cc:359] Metadata directory not provided
I20260923 23:18:15.281927 13291 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:18:15.282079 13291 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:18:15.283164 13291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790205259943518-25487-0/minicluster-data/master-0/data/instance:
uuid: "f72c290df23743cca22fa7754cd854f2"
format_stamp: "Formatted at 2026-09-23 23:18:15 on dist-test-slave-754n"
server_key: "fc023c18362726d9e73a9901c40b3f99"
server_key_iv: "a1725ba29ba0f300c23306f554fab31c"
server_key_version: "encryptionkey@0"
I20260923 23:18:15.283524 13291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790205259943518-25487-0/minicluster-data/master-0/wal/instance:
uuid: "f72c290df23743cca22fa7754cd854f2"
format_stamp: "Formatted at 2026-09-23 23:18:15 on dist-test-slave-754n"
server_key: "fc023c18362726d9e73a9901c40b3f99"
server_key_iv: "a1725ba29ba0f300c23306f554fab31c"
server_key_version: "encryptionkey@0"
I20260923 23:18:15.284829 13291 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 23:18:15.285495 13306 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:18:15.285674 13291 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 23:18:15.285758 13291 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790205259943518-25487-0/minicluster-data/master-0/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790205259943518-25487-0/minicluster-data/master-0/wal
uuid: "f72c290df23743cca22fa7754cd854f2"
format_stamp: "Formatted at 2026-09-23 23:18:15 on dist-test-slave-754n"
server_key: "fc023c18362726d9e73a9901c40b3f99"
server_key_iv: "a1725ba29ba0f300c23306f554fab31c"
server_key_version: "encryptionkey@0"
I20260923 23:18:15.285807 13291 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790205259943518-25487-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790205259943518-25487-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790205259943518-25487-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:18:15.301817 13291 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:18:15.303962 13291 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:18:15.304065 13291 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:18:15.307467 13291 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34377
I20260923 23:18:15.307502 13358 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34377 every 8 connection(s)
I20260923 23:18:15.307825 13291 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
I20260923 23:18:15.308668 13359 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:18:15.310819 13359 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f72c290df23743cca22fa7754cd854f2: Bootstrap starting.
I20260923 23:18:15.311106 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 13291
I20260923 23:18:15.311197 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790205259943518-25487-0/minicluster-data/master-0/wal/instance
I20260923 23:18:15.311405 25487 external_mini_cluster.cc:1428] Setting key d62816321c0d0cf3cd10b32bee2115b3
I20260923 23:18:15.311501 13359 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f72c290df23743cca22fa7754cd854f2: Neither blocks nor log segments found. Creating new log.
I20260923 23:18:15.311815 13359 log.cc:824] T 00000000000000000000000000000000 P f72c290df23743cca22fa7754cd854f2: Log is configured to *not* fsync() on all Append() calls
I20260923 23:18:15.312553 13359 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f72c290df23743cca22fa7754cd854f2: No bootstrap required, opened a new log
I20260923 23:18:15.314903 13359 raft_consensus.cc:348] T 00000000000000000000000000000000 P f72c290df23743cca22fa7754cd854f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f72c290df23743cca22fa7754cd854f2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34377 } }
I20260923 23:18:15.315032 13359 raft_consensus.cc:374] T 00000000000000000000000000000000 P f72c290df23743cca22fa7754cd854f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:18:15.315054 13359 raft_consensus.cc:729] T 00000000000000000000000000000000 P f72c290df23743cca22fa7754cd854f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f72c290df23743cca22fa7754cd854f2, State: Initialized, Role: FOLLOWER
I20260923 23:18:15.315179 13359 consensus_queue.cc:260] T 00000000000000000000000000000000 P f72c290df23743cca22fa7754cd854f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f72c290df23743cca22fa7754cd854f2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34377 } }
I20260923 23:18:15.315251 13359 raft_consensus.cc:388] T 00000000000000000000000000000000 P f72c290df23743cca22fa7754cd854f2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 23:18:15.315274 13359 raft_consensus.cc:482] T 00000000000000000000000000000000 P f72c290df23743cca22fa7754cd854f2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 23:18:15.315337 13359 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f72c290df23743cca22fa7754cd854f2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:18:15.316040 13359 raft_consensus.cc:504] T 00000000000000000000000000000000 P f72c290df23743cca22fa7754cd854f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f72c290df23743cca22fa7754cd854f2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34377 } }
I20260923 23:18:15.316156 13359 leader_election.cc:302] T 00000000000000000000000000000000 P f72c290df23743cca22fa7754cd854f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f72c290df23743cca22fa7754cd854f2; no voters:
I20260923 23:18:15.316304 13359 leader_election.cc:288] T 00000000000000000000000000000000 P f72c290df23743cca22fa7754cd854f2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 23:18:15.316381 13364 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f72c290df23743cca22fa7754cd854f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 23:18:15.316622 13364 raft_consensus.cc:686] T 00000000000000000000000000000000 P f72c290df23743cca22fa7754cd854f2 [term 1 LEADER]: Becoming Leader. State: Replica: f72c290df23743cca22fa7754cd854f2, State: Running, Role: LEADER
I20260923 23:18:15.316717 13364 consensus_queue.cc:237] T 00000000000000000000000000000000 P f72c290df23743cca22fa7754cd854f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f72c290df23743cca22fa7754cd854f2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34377 } }
I20260923 23:18:15.316776 13359 sys_catalog.cc:564] T 00000000000000000000000000000000 P f72c290df23743cca22fa7754cd854f2 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 23:18:15.317059 13365 sys_catalog.cc:455] T 00000000000000000000000000000000 P f72c290df23743cca22fa7754cd854f2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f72c290df23743cca22fa7754cd854f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f72c290df23743cca22fa7754cd854f2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34377 } } }
I20260923 23:18:15.317130 13365 sys_catalog.cc:458] T 00000000000000000000000000000000 P f72c290df23743cca22fa7754cd854f2 [sys.catalog]: This master's current role is: LEADER
I20260923 23:18:15.317263 13366 sys_catalog.cc:455] T 00000000000000000000000000000000 P f72c290df23743cca22fa7754cd854f2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f72c290df23743cca22fa7754cd854f2. Latest consensus state: current_term: 1 leader_uuid: "f72c290df23743cca22fa7754cd854f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f72c290df23743cca22fa7754cd854f2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34377 } } }
I20260923 23:18:15.317344 13366 sys_catalog.cc:458] T 00000000000000000000000000000000 P f72c290df23743cca22fa7754cd854f2 [sys.catalog]: This master's current role is: LEADER
I20260923 23:18:15.317467 13368 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 23:18:15.317606 13362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:15.311904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55907 (local address 127.0.0.1:34377)
0923 23:18:15.312111 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:15.312120 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:15.312150 (+ 30us) server_negotiation.cc:407] Connection header received
0923 23:18:15.312732 (+ 582us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:15.312740 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:15.312798 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:15.312937 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:15.313157 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:15.314522 (+ 1365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:15.315458 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:15.316139 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:15.316245 (+ 106us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:15.316878 (+ 633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:18:15.316881 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:18:15.316882 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 23:18:15.317084 (+ 202us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:18:15.317118 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 23:18:15.317129 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:18:15.317220 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:15.317374 (+ 154us) server_negotiation.cc:299] Negotiation successful
0923 23:18:15.317419 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":86,"threads_started":1}
I20260923 23:18:15.318322 13368 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 23:18:15.318508 25487 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 23:18:15.320044 13368 catalog_manager.cc:1347] Generated new cluster ID: aa062fe177d341198b23fd07edd14aee
I20260923 23:18:15.320097 13368 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 23:18:15.322845 13362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:15.319856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57517 (local address 127.0.0.1:34377)
0923 23:18:15.319906 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:15.319908 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:15.320163 (+ 255us) server_negotiation.cc:407] Connection header received
0923 23:18:15.320265 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:15.320266 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:15.320276 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:15.320318 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:15.320483 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:15.320920 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:15.321671 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:15.321893 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:15.321981 (+ 88us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:15.322606 (+ 625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:18:15.322607 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:18:15.322607 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 23:18:15.322611 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:18:15.322623 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 23:18:15.322627 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:18:15.322679 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:15.322738 (+ 59us) server_negotiation.cc:299] Negotiation successful
0923 23:18:15.322747 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260923 23:18:15.338001 13368 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 23:18:15.338593 13368 catalog_manager.cc:1504] Loading token signing keys...
I20260923 23:18:15.349893 13368 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f72c290df23743cca22fa7754cd854f2: Generated new TSK 0
I20260923 23:18:15.350052 13368 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 23:18:15.354517 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 13291
W20260923 23:18:15.358345 13276 connection.cc:535] client connection to 127.0.0.1:34377 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34377): Connection reset by peer (error 104)
2026-09-23T23:18:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1268 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790205259943518-25487-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 23:18:15 dist-test-slave-754n krb5kdc[13390](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 23:18:15 dist-test-slave-754n krb5kdc[13390](info): set up 2 sockets
Sep 23 23:18:15 dist-test-slave-754n krb5kdc[13390](info): commencing operation
krb5kdc: starting...
W20260923 23:18:16.367683 25487 mini_kdc.cc:121] Time spent starting KDC: real 0.996s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790205259943518-25487-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790205259943518-25487-0/krb5kdc/test-user.keytab.
Sep 23 23:18:16 dist-test-slave-754n krb5kdc[13390](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205496, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T23:18:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T23:18:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790205259943518-25487-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790205259943518-25487-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 23:18:16.474334 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790205259943518-25487-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790205259943518-25487-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790205259943518-25487-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37701
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790205259943518-25487-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:32775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37701
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:18:16.550527 13403 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:18:16.550715 13403 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:18:16.550737 13403 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:18:16.551757 13403 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 23:18:16.551788 13403 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:18:16.551802 13403 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:18:16.551815 13403 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 23:18:16.551826 13403 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 23:18:16.553174 13403 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:32775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790205259943518-25487-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790205259943518-25487-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37701
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790205259943518-25487-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37701
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13403
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790205259943518-25487-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:18:16.553340 13403 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:18:16.553488 13403 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:18:16.558014 13409 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:18:16.558051 13403 server_base.cc:1034] running on GCE node
W20260923 23:18:16.558071 13411 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:18:16.558022 13408 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:18:16.558354 13403 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:18:16.558615 13403 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:18:16.559764 13403 hybrid_clock.cc:648] HybridClock initialized: now 1790205496559758 us; error 32 us; skew 500 ppm
Sep 23 23:18:16 dist-test-slave-754n krb5kdc[13390](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205496, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:18:16.562088 13403 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 23:18:16.562511 13403 webserver.cc:460] Webserver started at http://127.0.0.1:34439/ using document root <none> and password file <none>
I20260923 23:18:16.562724 13403 fs_manager.cc:359] Metadata directory not provided
I20260923 23:18:16.562806 13403 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:18:16.562949 13403 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:18:16.563947 13403 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790205259943518-25487-0/minicluster-data/master-0/data/instance:
uuid: "a7f6e884192c4256a732e26ccf00afa9"
format_stamp: "Formatted at 2026-09-23 23:18:16 on dist-test-slave-754n"
server_key: "d28ae36f075f42a20f19f8ba51ffac5f"
server_key_iv: "cf85bd51f12de09c204460a75c7556e6"
server_key_version: "encryptionkey@0"
I20260923 23:18:16.564301 13403 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790205259943518-25487-0/minicluster-data/master-0/wal/instance:
uuid: "a7f6e884192c4256a732e26ccf00afa9"
format_stamp: "Formatted at 2026-09-23 23:18:16 on dist-test-slave-754n"
server_key: "d28ae36f075f42a20f19f8ba51ffac5f"
server_key_iv: "cf85bd51f12de09c204460a75c7556e6"
server_key_version: "encryptionkey@0"
I20260923 23:18:16.565492 13403 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 23:18:16.566133 13418 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:18:16.566300 13403 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 23:18:16.566407 13403 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790205259943518-25487-0/minicluster-data/master-0/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790205259943518-25487-0/minicluster-data/master-0/wal
uuid: "a7f6e884192c4256a732e26ccf00afa9"
format_stamp: "Formatted at 2026-09-23 23:18:16 on dist-test-slave-754n"
server_key: "d28ae36f075f42a20f19f8ba51ffac5f"
server_key_iv: "cf85bd51f12de09c204460a75c7556e6"
server_key_version: "encryptionkey@0"
I20260923 23:18:16.566483 13403 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790205259943518-25487-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790205259943518-25487-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790205259943518-25487-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:18:16.569620 13403 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:18:16.571740 13403 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:18:16.571933 13403 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:18:16.575407 13403 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37701
I20260923 23:18:16.575434 13470 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37701 every 8 connection(s)
I20260923 23:18:16.575706 13403 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
I20260923 23:18:16.576583 13471 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:18:16.577972 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 13403
I20260923 23:18:16.578076 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790205259943518-25487-0/minicluster-data/master-0/wal/instance
I20260923 23:18:16.578233 25487 external_mini_cluster.cc:1428] Setting key f8a0c9452d7568882533d2907bd58675
I20260923 23:18:16.578560 13471 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7f6e884192c4256a732e26ccf00afa9: Bootstrap starting.
I20260923 23:18:16.579180 13471 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a7f6e884192c4256a732e26ccf00afa9: Neither blocks nor log segments found. Creating new log.
I20260923 23:18:16.579468 13471 log.cc:824] T 00000000000000000000000000000000 P a7f6e884192c4256a732e26ccf00afa9: Log is configured to *not* fsync() on all Append() calls
I20260923 23:18:16.580192 13471 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7f6e884192c4256a732e26ccf00afa9: No bootstrap required, opened a new log
I20260923 23:18:16.580623 13474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:16.578641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56445 (local address 127.0.0.1:37701)
0923 23:18:16.578820 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:16.578830 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:16.578862 (+ 32us) server_negotiation.cc:407] Connection header received
0923 23:18:16.579404 (+ 542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:16.579413 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:16.579443 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:16.579590 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:16.580184 (+ 594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:18:16.580187 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:18:16.580187 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 23:18:16.580191 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:18:16.580248 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 23:18:16.580261 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:18:16.580355 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:16.580423 (+ 68us) server_negotiation.cc:299] Negotiation successful
0923 23:18:16.580457 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":88,"threads_started":1}
I20260923 23:18:16.581612 25487 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 23:18:16.582316 13471 raft_consensus.cc:348] T 00000000000000000000000000000000 P a7f6e884192c4256a732e26ccf00afa9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7f6e884192c4256a732e26ccf00afa9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37701 } }
I20260923 23:18:16.582491 13471 raft_consensus.cc:374] T 00000000000000000000000000000000 P a7f6e884192c4256a732e26ccf00afa9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:18:16.582523 13471 raft_consensus.cc:729] T 00000000000000000000000000000000 P a7f6e884192c4256a732e26ccf00afa9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7f6e884192c4256a732e26ccf00afa9, State: Initialized, Role: FOLLOWER
I20260923 23:18:16.582639 13471 consensus_queue.cc:260] T 00000000000000000000000000000000 P a7f6e884192c4256a732e26ccf00afa9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7f6e884192c4256a732e26ccf00afa9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37701 } }
I20260923 23:18:16.582738 13471 raft_consensus.cc:388] T 00000000000000000000000000000000 P a7f6e884192c4256a732e26ccf00afa9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 23:18:16.582783 13471 raft_consensus.cc:482] T 00000000000000000000000000000000 P a7f6e884192c4256a732e26ccf00afa9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 23:18:16.582840 13471 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a7f6e884192c4256a732e26ccf00afa9 [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:18:16.583527 13471 raft_consensus.cc:504] T 00000000000000000000000000000000 P a7f6e884192c4256a732e26ccf00afa9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7f6e884192c4256a732e26ccf00afa9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37701 } }
I20260923 23:18:16.583659 13471 leader_election.cc:302] T 00000000000000000000000000000000 P a7f6e884192c4256a732e26ccf00afa9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a7f6e884192c4256a732e26ccf00afa9; no voters:
I20260923 23:18:16.583813 13471 leader_election.cc:288] T 00000000000000000000000000000000 P a7f6e884192c4256a732e26ccf00afa9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 23:18:16.583889 13481 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a7f6e884192c4256a732e26ccf00afa9 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 23:18:16.584036 13481 raft_consensus.cc:686] T 00000000000000000000000000000000 P a7f6e884192c4256a732e26ccf00afa9 [term 1 LEADER]: Becoming Leader. State: Replica: a7f6e884192c4256a732e26ccf00afa9, State: Running, Role: LEADER
I20260923 23:18:16.584127 13471 sys_catalog.cc:564] T 00000000000000000000000000000000 P a7f6e884192c4256a732e26ccf00afa9 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 23:18:16.584174 13481 consensus_queue.cc:237] T 00000000000000000000000000000000 P a7f6e884192c4256a732e26ccf00afa9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7f6e884192c4256a732e26ccf00afa9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37701 } }
I20260923 23:18:16.584277 13474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:16.583159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35961 (local address 127.0.0.1:37701)
0923 23:18:16.583196 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:16.583198 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:16.583277 (+ 79us) server_negotiation.cc:407] Connection header received
0923 23:18:16.583342 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:16.583342 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:16.583345 (+ 3us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:16.583393 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:16.583968 (+ 575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:18:16.583970 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:18:16.583970 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 23:18:16.583975 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:18:16.583987 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 23:18:16.583992 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:18:16.584039 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:16.584212 (+ 173us) server_negotiation.cc:299] Negotiation successful
0923 23:18:16.584225 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260923 23:18:16.584590 13483 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7f6e884192c4256a732e26ccf00afa9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a7f6e884192c4256a732e26ccf00afa9. Latest consensus state: current_term: 1 leader_uuid: "a7f6e884192c4256a732e26ccf00afa9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7f6e884192c4256a732e26ccf00afa9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37701 } } }
I20260923 23:18:16.584609 13482 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7f6e884192c4256a732e26ccf00afa9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a7f6e884192c4256a732e26ccf00afa9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7f6e884192c4256a732e26ccf00afa9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37701 } } }
I20260923 23:18:16.584669 13483 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7f6e884192c4256a732e26ccf00afa9 [sys.catalog]: This master's current role is: LEADER
I20260923 23:18:16.584692 13482 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7f6e884192c4256a732e26ccf00afa9 [sys.catalog]: This master's current role is: LEADER
I20260923 23:18:16.584877 13487 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 23:18:16.585371 13487 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 23:18:16.586776 13487 catalog_manager.cc:1347] Generated new cluster ID: 663946c18ff64f98af0f006416759210
I20260923 23:18:16.586824 13487 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 23:18:16.619892 13487 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 23:18:16.620039 13487 catalog_manager.cc:1504] Loading token signing keys...
I20260923 23:18:16.627516 13487 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a7f6e884192c4256a732e26ccf00afa9: Generated new TSK 0
I20260923 23:18:16.627672 13487 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 23:18:16.636737 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 13403
2026-09-23T23:18:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1282 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790205259943518-25487-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 23:18:16 dist-test-slave-754n krb5kdc[13502](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 23:18:16 dist-test-slave-754n krb5kdc[13502](info): set up 2 sockets
Sep 23 23:18:16 dist-test-slave-754n krb5kdc[13502](info): commencing operation
krb5kdc: starting...
W20260923 23:18:17.647785 25487 mini_kdc.cc:121] Time spent starting KDC: real 0.993s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790205259943518-25487-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790205259943518-25487-0/krb5kdc/test-user.keytab.
Sep 23 23:18:17 dist-test-slave-754n krb5kdc[13502](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205497, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T23:18:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T23:18:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790205259943518-25487-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790205259943518-25487-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 23:18:17.758504 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790205259943518-25487-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790205259943518-25487-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790205259943518-25487-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36747
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790205259943518-25487-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36747
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:18:17.836376 13515 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:18:17.836581 13515 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:18:17.836618 13515 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:18:17.837741 13515 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 23:18:17.837838 13515 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:18:17.837867 13515 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:18:17.837896 13515 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 23:18:17.837919 13515 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 23:18:17.839323 13515 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790205259943518-25487-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790205259943518-25487-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36747
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790205259943518-25487-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36747
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13515
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790205259943518-25487-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:18:17.839558 13515 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:18:17.839749 13515 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:18:17.844285 13523 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:18:17.844292 13520 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:18:17.844302 13521 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:18:17.844547 13515 server_base.cc:1034] running on GCE node
I20260923 23:18:17.844753 13515 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:18:17.845021 13515 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:18:17.846174 13515 hybrid_clock.cc:648] HybridClock initialized: now 1790205497846162 us; error 28 us; skew 500 ppm
Sep 23 23:18:17 dist-test-slave-754n krb5kdc[13502](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205497, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:18:17.848474 13515 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 23:18:17.848888 13515 webserver.cc:460] Webserver started at http://127.0.0.1:41767/ using document root <none> and password file <none>
I20260923 23:18:17.849102 13515 fs_manager.cc:359] Metadata directory not provided
I20260923 23:18:17.849174 13515 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:18:17.849298 13515 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:18:17.850297 13515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790205259943518-25487-0/minicluster-data/master-0/data/instance:
uuid: "7d3bcf8ea9294e9fa73b1263b7f22b08"
format_stamp: "Formatted at 2026-09-23 23:18:17 on dist-test-slave-754n"
server_key: "733b63be8a79fd92ebac81a33c932902"
server_key_iv: "16b2ffe2808531c235a722aa9cc8d8cd"
server_key_version: "encryptionkey@0"
I20260923 23:18:17.850663 13515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790205259943518-25487-0/minicluster-data/master-0/wal/instance:
uuid: "7d3bcf8ea9294e9fa73b1263b7f22b08"
format_stamp: "Formatted at 2026-09-23 23:18:17 on dist-test-slave-754n"
server_key: "733b63be8a79fd92ebac81a33c932902"
server_key_iv: "16b2ffe2808531c235a722aa9cc8d8cd"
server_key_version: "encryptionkey@0"
I20260923 23:18:17.852005 13515 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260923 23:18:17.852706 13530 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:18:17.852876 13515 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 23:18:17.852965 13515 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790205259943518-25487-0/minicluster-data/master-0/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790205259943518-25487-0/minicluster-data/master-0/wal
uuid: "7d3bcf8ea9294e9fa73b1263b7f22b08"
format_stamp: "Formatted at 2026-09-23 23:18:17 on dist-test-slave-754n"
server_key: "733b63be8a79fd92ebac81a33c932902"
server_key_iv: "16b2ffe2808531c235a722aa9cc8d8cd"
server_key_version: "encryptionkey@0"
I20260923 23:18:17.853044 13515 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790205259943518-25487-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790205259943518-25487-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790205259943518-25487-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:18:17.872960 13515 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:18:17.874840 13515 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:18:17.875003 13515 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:18:17.878728 13515 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36747
I20260923 23:18:17.878768 13582 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36747 every 8 connection(s)
I20260923 23:18:17.879129 13515 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
I20260923 23:18:17.879932 13583 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:18:17.882122 13583 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d3bcf8ea9294e9fa73b1263b7f22b08: Bootstrap starting.
I20260923 23:18:17.882436 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 13515
I20260923 23:18:17.882547 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790205259943518-25487-0/minicluster-data/master-0/wal/instance
I20260923 23:18:17.882727 25487 external_mini_cluster.cc:1428] Setting key 59114994a053d7b8c186ab8916b90328
I20260923 23:18:17.882791 13583 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7d3bcf8ea9294e9fa73b1263b7f22b08: Neither blocks nor log segments found. Creating new log.
I20260923 23:18:17.883102 13583 log.cc:824] T 00000000000000000000000000000000 P 7d3bcf8ea9294e9fa73b1263b7f22b08: Log is configured to *not* fsync() on all Append() calls
I20260923 23:18:17.883793 13583 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d3bcf8ea9294e9fa73b1263b7f22b08: No bootstrap required, opened a new log
I20260923 23:18:17.886098 13583 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7d3bcf8ea9294e9fa73b1263b7f22b08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d3bcf8ea9294e9fa73b1263b7f22b08" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36747 } }
I20260923 23:18:17.886260 13583 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7d3bcf8ea9294e9fa73b1263b7f22b08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:18:17.886300 13583 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7d3bcf8ea9294e9fa73b1263b7f22b08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d3bcf8ea9294e9fa73b1263b7f22b08, State: Initialized, Role: FOLLOWER
I20260923 23:18:17.886415 13583 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7d3bcf8ea9294e9fa73b1263b7f22b08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d3bcf8ea9294e9fa73b1263b7f22b08" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36747 } }
I20260923 23:18:17.886479 13583 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7d3bcf8ea9294e9fa73b1263b7f22b08 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 23:18:17.886504 13583 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7d3bcf8ea9294e9fa73b1263b7f22b08 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 23:18:17.886544 13583 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7d3bcf8ea9294e9fa73b1263b7f22b08 [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:18:17.887082 13583 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7d3bcf8ea9294e9fa73b1263b7f22b08 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d3bcf8ea9294e9fa73b1263b7f22b08" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36747 } }
I20260923 23:18:17.887166 13583 leader_election.cc:302] T 00000000000000000000000000000000 P 7d3bcf8ea9294e9fa73b1263b7f22b08 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7d3bcf8ea9294e9fa73b1263b7f22b08; no voters:
I20260923 23:18:17.887357 13583 leader_election.cc:288] T 00000000000000000000000000000000 P 7d3bcf8ea9294e9fa73b1263b7f22b08 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 23:18:17.887378 13588 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7d3bcf8ea9294e9fa73b1263b7f22b08 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 23:18:17.887557 13588 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7d3bcf8ea9294e9fa73b1263b7f22b08 [term 1 LEADER]: Becoming Leader. State: Replica: 7d3bcf8ea9294e9fa73b1263b7f22b08, State: Running, Role: LEADER
I20260923 23:18:17.887593 13583 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7d3bcf8ea9294e9fa73b1263b7f22b08 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 23:18:17.887656 13588 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7d3bcf8ea9294e9fa73b1263b7f22b08 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d3bcf8ea9294e9fa73b1263b7f22b08" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36747 } }
I20260923 23:18:17.888018 13590 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d3bcf8ea9294e9fa73b1263b7f22b08 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7d3bcf8ea9294e9fa73b1263b7f22b08" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d3bcf8ea9294e9fa73b1263b7f22b08" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36747 } } }
I20260923 23:18:17.888007 13588 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d3bcf8ea9294e9fa73b1263b7f22b08 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7d3bcf8ea9294e9fa73b1263b7f22b08. Latest consensus state: current_term: 1 leader_uuid: "7d3bcf8ea9294e9fa73b1263b7f22b08" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d3bcf8ea9294e9fa73b1263b7f22b08" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36747 } } }
I20260923 23:18:17.888154 13590 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d3bcf8ea9294e9fa73b1263b7f22b08 [sys.catalog]: This master's current role is: LEADER
I20260923 23:18:17.888160 13588 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d3bcf8ea9294e9fa73b1263b7f22b08 [sys.catalog]: This master's current role is: LEADER
I20260923 23:18:17.888429 13597 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 23:18:17.888890 13597 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 23 23:18:17 dist-test-slave-754n krb5kdc[13502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205497, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260923 23:18:17.890424 13597 catalog_manager.cc:1347] Generated new cluster ID: 68178e2b61264da7b89b9fde45535b54
I20260923 23:18:17.890475 13597 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 23:18:17.893769 13586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:17.883188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37457 (local address 127.0.0.1:36747)
0923 23:18:17.883362 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:17.883371 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:17.883401 (+ 30us) server_negotiation.cc:407] Connection header received
0923 23:18:17.883901 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:17.883910 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:17.883973 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:17.884083 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:17.884872 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:17.885707 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:17.886741 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:17.887123 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:17.887252 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:17.890174 (+ 2922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:18:17.890177 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:18:17.890178 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:18:17.890189 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:18:17.892189 (+ 2000us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:17.892554 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:17.892557 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:17.892557 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:17.892660 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:17.892857 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:17.892858 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:17.892858 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:17.893151 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:18:17.893308 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:17.893416 (+ 108us) server_negotiation.cc:299] Negotiation successful
0923 23:18:17.893558 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":75,"threads_started":1}
I20260923 23:18:17.894660 25487 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 23:18:17.903211 13586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:17.896178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54985 (local address 127.0.0.1:36747)
0923 23:18:17.896230 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:17.896232 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:17.896318 (+ 86us) server_negotiation.cc:407] Connection header received
0923 23:18:17.896382 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:17.896383 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:17.896391 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:17.896440 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:17.897028 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:17.898043 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:17.899056 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:17.899366 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:17.899416 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:17.900701 (+ 1285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:18:17.900702 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:18:17.900703 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:18:17.900709 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:18:17.902175 (+ 1466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:17.902544 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:17.902545 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:17.902545 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:17.902595 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:17.902799 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:17.902800 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:17.902800 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:17.902896 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:18:17.902969 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:17.903081 (+ 112us) server_negotiation.cc:299] Negotiation successful
0923 23:18:17.903138 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260923 23:18:17.906872 13597 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 23:18:17.907310 13597 catalog_manager.cc:1504] Loading token signing keys...
I20260923 23:18:17.914884 13597 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7d3bcf8ea9294e9fa73b1263b7f22b08: Generated new TSK 0
I20260923 23:18:17.915076 13597 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 23:18:17.921789 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 13515
2026-09-23T23:18:17Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1284 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790205259943518-25487-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 23:18:17 dist-test-slave-754n krb5kdc[13614](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 23:18:17 dist-test-slave-754n krb5kdc[13614](info): set up 2 sockets
Sep 23 23:18:17 dist-test-slave-754n krb5kdc[13614](info): commencing operation
krb5kdc: starting...
W20260923 23:18:18.931740 25487 mini_kdc.cc:121] Time spent starting KDC: real 0.992s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790205259943518-25487-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790205259943518-25487-0/krb5kdc/test-user.keytab.
Sep 23 23:18:19 dist-test-slave-754n krb5kdc[13614](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205499, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T23:18:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T23:18:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790205259943518-25487-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790205259943518-25487-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 23:18:19.038635 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790205259943518-25487-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790205259943518-25487-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790205259943518-25487-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41899
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790205259943518-25487-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41899
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:18:19.113759 13627 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:18:19.113955 13627 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:18:19.113989 13627 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:18:19.115046 13627 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 23:18:19.115082 13627 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:18:19.115094 13627 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:18:19.115105 13627 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 23:18:19.115149 13627 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 23:18:19.116456 13627 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790205259943518-25487-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790205259943518-25487-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41899
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790205259943518-25487-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41899
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13627
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790205259943518-25487-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:18:19.116665 13627 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:18:19.116866 13627 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:18:19.121006 13632 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:18:19.120985 13633 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:18:19.121186 13627 server_base.cc:1034] running on GCE node
W20260923 23:18:19.121196 13635 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:18:19.121434 13627 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:18:19.121714 13627 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:18:19.122850 13627 hybrid_clock.cc:648] HybridClock initialized: now 1790205499122841 us; error 31 us; skew 500 ppm
Sep 23 23:18:19 dist-test-slave-754n krb5kdc[13614](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205499, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:18:19.125164 13627 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 23:18:19.125571 13627 webserver.cc:460] Webserver started at http://127.0.0.1:45227/ using document root <none> and password file <none>
I20260923 23:18:19.125761 13627 fs_manager.cc:359] Metadata directory not provided
I20260923 23:18:19.125800 13627 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:18:19.125878 13627 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:18:19.126744 13627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790205259943518-25487-0/minicluster-data/master-0/data/instance:
uuid: "1479b94c96664e9586082aad2db59798"
format_stamp: "Formatted at 2026-09-23 23:18:19 on dist-test-slave-754n"
server_key: "bf9b6ad54faf97e04af19c76d543a4a2"
server_key_iv: "caa47d472b0c053538874222fee286a2"
server_key_version: "encryptionkey@0"
I20260923 23:18:19.126999 13627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790205259943518-25487-0/minicluster-data/master-0/wal/instance:
uuid: "1479b94c96664e9586082aad2db59798"
format_stamp: "Formatted at 2026-09-23 23:18:19 on dist-test-slave-754n"
server_key: "bf9b6ad54faf97e04af19c76d543a4a2"
server_key_iv: "caa47d472b0c053538874222fee286a2"
server_key_version: "encryptionkey@0"
I20260923 23:18:19.128201 13627 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260923 23:18:19.128839 13642 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:18:19.128979 13627 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 23:18:19.129076 13627 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790205259943518-25487-0/minicluster-data/master-0/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790205259943518-25487-0/minicluster-data/master-0/wal
uuid: "1479b94c96664e9586082aad2db59798"
format_stamp: "Formatted at 2026-09-23 23:18:19 on dist-test-slave-754n"
server_key: "bf9b6ad54faf97e04af19c76d543a4a2"
server_key_iv: "caa47d472b0c053538874222fee286a2"
server_key_version: "encryptionkey@0"
I20260923 23:18:19.129135 13627 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790205259943518-25487-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790205259943518-25487-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790205259943518-25487-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:18:19.141227 13627 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:18:19.143224 13627 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:18:19.143360 13627 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:18:19.146755 13627 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41899
I20260923 23:18:19.146790 13694 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41899 every 8 connection(s)
I20260923 23:18:19.147073 13627 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
I20260923 23:18:19.147804 13695 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:18:19.149767 13695 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1479b94c96664e9586082aad2db59798: Bootstrap starting.
I20260923 23:18:19.150244 13695 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1479b94c96664e9586082aad2db59798: Neither blocks nor log segments found. Creating new log.
I20260923 23:18:19.150470 13695 log.cc:824] T 00000000000000000000000000000000 P 1479b94c96664e9586082aad2db59798: Log is configured to *not* fsync() on all Append() calls
I20260923 23:18:19.151041 13695 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1479b94c96664e9586082aad2db59798: No bootstrap required, opened a new log
I20260923 23:18:19.152503 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 13627
I20260923 23:18:19.152635 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790205259943518-25487-0/minicluster-data/master-0/wal/instance
I20260923 23:18:19.152861 25487 external_mini_cluster.cc:1428] Setting key 95b140ff6585bdca60dbb65cff698e88
I20260923 23:18:19.152841 13695 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1479b94c96664e9586082aad2db59798 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1479b94c96664e9586082aad2db59798" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41899 } }
I20260923 23:18:19.152989 13695 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1479b94c96664e9586082aad2db59798 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:18:19.153030 13695 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1479b94c96664e9586082aad2db59798 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1479b94c96664e9586082aad2db59798, State: Initialized, Role: FOLLOWER
I20260923 23:18:19.153191 13695 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1479b94c96664e9586082aad2db59798 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1479b94c96664e9586082aad2db59798" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41899 } }
I20260923 23:18:19.153285 13695 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1479b94c96664e9586082aad2db59798 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 23:18:19.153324 13695 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1479b94c96664e9586082aad2db59798 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 23:18:19.153370 13695 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1479b94c96664e9586082aad2db59798 [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:18:19.153903 13695 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1479b94c96664e9586082aad2db59798 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1479b94c96664e9586082aad2db59798" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41899 } }
I20260923 23:18:19.153986 13695 leader_election.cc:302] T 00000000000000000000000000000000 P 1479b94c96664e9586082aad2db59798 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1479b94c96664e9586082aad2db59798; no voters:
I20260923 23:18:19.154175 13695 leader_election.cc:288] T 00000000000000000000000000000000 P 1479b94c96664e9586082aad2db59798 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 23:18:19.154199 13700 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1479b94c96664e9586082aad2db59798 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 23:18:19.154310 13700 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1479b94c96664e9586082aad2db59798 [term 1 LEADER]: Becoming Leader. State: Replica: 1479b94c96664e9586082aad2db59798, State: Running, Role: LEADER
I20260923 23:18:19.154387 13700 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1479b94c96664e9586082aad2db59798 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1479b94c96664e9586082aad2db59798" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41899 } }
I20260923 23:18:19.154426 13695 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1479b94c96664e9586082aad2db59798 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 23:18:19.154709 13700 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1479b94c96664e9586082aad2db59798 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1479b94c96664e9586082aad2db59798. Latest consensus state: current_term: 1 leader_uuid: "1479b94c96664e9586082aad2db59798" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1479b94c96664e9586082aad2db59798" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41899 } } }
I20260923 23:18:19.154793 13700 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1479b94c96664e9586082aad2db59798 [sys.catalog]: This master's current role is: LEADER
I20260923 23:18:19.154721 13702 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1479b94c96664e9586082aad2db59798 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1479b94c96664e9586082aad2db59798" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1479b94c96664e9586082aad2db59798" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41899 } } }
I20260923 23:18:19.154915 13702 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1479b94c96664e9586082aad2db59798 [sys.catalog]: This master's current role is: LEADER
I20260923 23:18:19.155580 13705 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 23:18:19.156097 13705 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 23:18:19.157456 13705 catalog_manager.cc:1347] Generated new cluster ID: 6037d6c599274cb1bc176332fe3bec60
I20260923 23:18:19.157500 13705 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 23:18:19.160744 13699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:19.153436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38803 (local address 127.0.0.1:41899)
0923 23:18:19.153577 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:19.153585 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:19.153613 (+ 28us) server_negotiation.cc:407] Connection header received
0923 23:18:19.153896 (+ 283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:19.153902 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:19.153940 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:19.154087 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:19.154329 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:19.155519 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:19.156591 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:19.157106 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:19.157226 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:19.160089 (+ 2863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:18:19.160093 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:18:19.160093 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 23:18:19.160294 (+ 201us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:18:19.160329 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 23:18:19.160342 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:18:19.160407 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:19.160476 (+ 69us) server_negotiation.cc:299] Negotiation successful
0923 23:18:19.160550 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":74,"threads_started":1}
I20260923 23:18:19.161712 25487 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 23:18:19.169185 13705 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 23:18:19.169770 13705 catalog_manager.cc:1504] Loading token signing keys...
I20260923 23:18:19.172510 13699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:19.169135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33245 (local address 127.0.0.1:41899)
0923 23:18:19.169165 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:19.169166 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:19.169353 (+ 187us) server_negotiation.cc:407] Connection header received
0923 23:18:19.169395 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:19.169396 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:19.169406 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:19.169460 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:19.169571 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:19.170337 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:19.171293 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:19.171603 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:19.171680 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:19.172277 (+ 597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:18:19.172278 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:18:19.172278 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 23:18:19.172283 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:18:19.172297 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 23:18:19.172301 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:18:19.172358 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:19.172406 (+ 48us) server_negotiation.cc:299] Negotiation successful
0923 23:18:19.172440 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":17,"server-negotiator.queue_time_us":17}
I20260923 23:18:19.173985 13705 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1479b94c96664e9586082aad2db59798: Generated new TSK 0
I20260923 23:18:19.174129 13705 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 23:18:19.177600 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 13627
2026-09-23T23:18:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1255 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790205259943518-25487-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 23:18:19 dist-test-slave-754n krb5kdc[13726](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 23:18:19 dist-test-slave-754n krb5kdc[13726](info): set up 2 sockets
Sep 23 23:18:19 dist-test-slave-754n krb5kdc[13726](info): commencing operation
krb5kdc: starting...
W20260923 23:18:20.175989 25487 mini_kdc.cc:121] Time spent starting KDC: real 0.981s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790205259943518-25487-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790205259943518-25487-0/krb5kdc/test-user.keytab.
Sep 23 23:18:20 dist-test-slave-754n krb5kdc[13726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205500, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T23:18:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T23:18:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790205259943518-25487-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790205259943518-25487-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 23:18:20.284606 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790205259943518-25487-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790205259943518-25487-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790205259943518-25487-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34083
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790205259943518-25487-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34083
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:18:20.362147 13739 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:18:20.362332 13739 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:18:20.362354 13739 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:18:20.363430 13739 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 23:18:20.363464 13739 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:18:20.363479 13739 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:18:20.363492 13739 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 23:18:20.363504 13739 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 23:18:20.364938 13739 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790205259943518-25487-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790205259943518-25487-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34083
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790205259943518-25487-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34083
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13739
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790205259943518-25487-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:18:20.365113 13739 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:18:20.365269 13739 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260923 23:18:20.369820 13739 server_base.cc:1034] running on GCE node
W20260923 23:18:20.369813 13744 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:18:20.369804 13747 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:18:20.369798 13745 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:18:20.370273 13739 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:18:20.370553 13739 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:18:20.371717 13739 hybrid_clock.cc:648] HybridClock initialized: now 1790205500371700 us; error 37 us; skew 500 ppm
Sep 23 23:18:20 dist-test-slave-754n krb5kdc[13726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205500, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:18:20.379930 13739 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 23:18:20.380336 13739 webserver.cc:460] Webserver started at http://127.0.0.1:35171/ using document root <none> and password file <none>
I20260923 23:18:20.380573 13739 fs_manager.cc:359] Metadata directory not provided
I20260923 23:18:20.380648 13739 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:18:20.380789 13739 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:18:20.381807 13739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790205259943518-25487-0/minicluster-data/master-0/data/instance:
uuid: "e31033b32a1e43eeb6d524ed9b1a3282"
format_stamp: "Formatted at 2026-09-23 23:18:20 on dist-test-slave-754n"
server_key: "1502b0bb7a4466d1f1be770acd3db871"
server_key_iv: "e40601461bd7b445113e6d6c105a215c"
server_key_version: "encryptionkey@0"
I20260923 23:18:20.382148 13739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790205259943518-25487-0/minicluster-data/master-0/wal/instance:
uuid: "e31033b32a1e43eeb6d524ed9b1a3282"
format_stamp: "Formatted at 2026-09-23 23:18:20 on dist-test-slave-754n"
server_key: "1502b0bb7a4466d1f1be770acd3db871"
server_key_iv: "e40601461bd7b445113e6d6c105a215c"
server_key_version: "encryptionkey@0"
I20260923 23:18:20.383443 13739 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260923 23:18:20.384179 13754 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:18:20.384325 13739 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 23:18:20.384414 13739 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790205259943518-25487-0/minicluster-data/master-0/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790205259943518-25487-0/minicluster-data/master-0/wal
uuid: "e31033b32a1e43eeb6d524ed9b1a3282"
format_stamp: "Formatted at 2026-09-23 23:18:20 on dist-test-slave-754n"
server_key: "1502b0bb7a4466d1f1be770acd3db871"
server_key_iv: "e40601461bd7b445113e6d6c105a215c"
server_key_version: "encryptionkey@0"
I20260923 23:18:20.384478 13739 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790205259943518-25487-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790205259943518-25487-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790205259943518-25487-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:18:20.387485 13739 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:18:20.389423 13739 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:18:20.389568 13739 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:18:20.393121 13739 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34083
I20260923 23:18:20.393149 13806 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34083 every 8 connection(s)
I20260923 23:18:20.393468 13739 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
I20260923 23:18:20.394382 13807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:18:20.396523 13807 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e31033b32a1e43eeb6d524ed9b1a3282: Bootstrap starting.
I20260923 23:18:20.397064 13807 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e31033b32a1e43eeb6d524ed9b1a3282: Neither blocks nor log segments found. Creating new log.
I20260923 23:18:20.397323 13807 log.cc:824] T 00000000000000000000000000000000 P e31033b32a1e43eeb6d524ed9b1a3282: Log is configured to *not* fsync() on all Append() calls
I20260923 23:18:20.397943 13807 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e31033b32a1e43eeb6d524ed9b1a3282: No bootstrap required, opened a new log
I20260923 23:18:20.398518 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 13739
I20260923 23:18:20.398633 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790205259943518-25487-0/minicluster-data/master-0/wal/instance
I20260923 23:18:20.398816 25487 external_mini_cluster.cc:1428] Setting key 3f289a91506e4cfbdb945d20e717925b
I20260923 23:18:20.400048 13807 raft_consensus.cc:348] T 00000000000000000000000000000000 P e31033b32a1e43eeb6d524ed9b1a3282 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e31033b32a1e43eeb6d524ed9b1a3282" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34083 } }
I20260923 23:18:20.400220 13807 raft_consensus.cc:374] T 00000000000000000000000000000000 P e31033b32a1e43eeb6d524ed9b1a3282 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:18:20.400259 13807 raft_consensus.cc:729] T 00000000000000000000000000000000 P e31033b32a1e43eeb6d524ed9b1a3282 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e31033b32a1e43eeb6d524ed9b1a3282, State: Initialized, Role: FOLLOWER
I20260923 23:18:20.400770 13807 consensus_queue.cc:260] T 00000000000000000000000000000000 P e31033b32a1e43eeb6d524ed9b1a3282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e31033b32a1e43eeb6d524ed9b1a3282" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34083 } }
I20260923 23:18:20.400940 13807 raft_consensus.cc:388] T 00000000000000000000000000000000 P e31033b32a1e43eeb6d524ed9b1a3282 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 23:18:20.400983 13807 raft_consensus.cc:482] T 00000000000000000000000000000000 P e31033b32a1e43eeb6d524ed9b1a3282 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 23:18:20.401043 13807 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e31033b32a1e43eeb6d524ed9b1a3282 [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:18:20.401459 13811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:20.399283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53269 (local address 127.0.0.1:34083)
0923 23:18:20.399476 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:20.399487 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:20.399519 (+ 32us) server_negotiation.cc:407] Connection header received
0923 23:18:20.400007 (+ 488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:20.400015 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:20.400050 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:20.400205 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:20.400949 (+ 744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:18:20.400953 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:18:20.400954 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 23:18:20.400959 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:18:20.401010 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 23:18:20.401027 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:18:20.401125 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:20.401198 (+ 73us) server_negotiation.cc:299] Negotiation successful
0923 23:18:20.401241 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":92,"threads_started":1}
I20260923 23:18:20.402172 13807 raft_consensus.cc:504] T 00000000000000000000000000000000 P e31033b32a1e43eeb6d524ed9b1a3282 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e31033b32a1e43eeb6d524ed9b1a3282" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34083 } }
I20260923 23:18:20.402294 13807 leader_election.cc:302] T 00000000000000000000000000000000 P e31033b32a1e43eeb6d524ed9b1a3282 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e31033b32a1e43eeb6d524ed9b1a3282; no voters:
I20260923 23:18:20.402388 25487 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 23:18:20.402480 13807 leader_election.cc:288] T 00000000000000000000000000000000 P e31033b32a1e43eeb6d524ed9b1a3282 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 23:18:20.402518 13812 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e31033b32a1e43eeb6d524ed9b1a3282 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 23:18:20.402626 13812 raft_consensus.cc:686] T 00000000000000000000000000000000 P e31033b32a1e43eeb6d524ed9b1a3282 [term 1 LEADER]: Becoming Leader. State: Replica: e31033b32a1e43eeb6d524ed9b1a3282, State: Running, Role: LEADER
I20260923 23:18:20.402841 13807 sys_catalog.cc:564] T 00000000000000000000000000000000 P e31033b32a1e43eeb6d524ed9b1a3282 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 23:18:20.402837 13812 consensus_queue.cc:237] T 00000000000000000000000000000000 P e31033b32a1e43eeb6d524ed9b1a3282 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e31033b32a1e43eeb6d524ed9b1a3282" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34083 } }
I20260923 23:18:20.403393 13813 sys_catalog.cc:455] T 00000000000000000000000000000000 P e31033b32a1e43eeb6d524ed9b1a3282 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e31033b32a1e43eeb6d524ed9b1a3282" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e31033b32a1e43eeb6d524ed9b1a3282" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34083 } } }
I20260923 23:18:20.403550 13813 sys_catalog.cc:458] T 00000000000000000000000000000000 P e31033b32a1e43eeb6d524ed9b1a3282 [sys.catalog]: This master's current role is: LEADER
I20260923 23:18:20.403976 13814 sys_catalog.cc:455] T 00000000000000000000000000000000 P e31033b32a1e43eeb6d524ed9b1a3282 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e31033b32a1e43eeb6d524ed9b1a3282. Latest consensus state: current_term: 1 leader_uuid: "e31033b32a1e43eeb6d524ed9b1a3282" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e31033b32a1e43eeb6d524ed9b1a3282" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34083 } } }
I20260923 23:18:20.404033 13826 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 23:18:20.404084 13814 sys_catalog.cc:458] T 00000000000000000000000000000000 P e31033b32a1e43eeb6d524ed9b1a3282 [sys.catalog]: This master's current role is: LEADER
I20260923 23:18:20.404577 13826 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 23:18:20.405972 13811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:20.404343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39079 (local address 127.0.0.1:34083)
0923 23:18:20.404385 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:20.404387 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:20.404599 (+ 212us) server_negotiation.cc:407] Connection header received
0923 23:18:20.404713 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:20.404714 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:20.404719 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:20.404779 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:20.405544 (+ 765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:18:20.405545 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:18:20.405545 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 23:18:20.405549 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:18:20.405562 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 23:18:20.405566 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:18:20.405595 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:20.405883 (+ 288us) server_negotiation.cc:299] Negotiation successful
0923 23:18:20.405899 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260923 23:18:20.406342 13826 catalog_manager.cc:1347] Generated new cluster ID: 03539fb1282a45099160fe402b617041
I20260923 23:18:20.406387 13826 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 23:18:20.415598 13826 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 23:18:20.415747 13826 catalog_manager.cc:1504] Loading token signing keys...
I20260923 23:18:20.418764 13826 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e31033b32a1e43eeb6d524ed9b1a3282: Generated new TSK 0
I20260923 23:18:20.418919 13826 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 23:18:20.424397 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 13739
2026-09-23T23:18:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1246 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790205259943518-25487-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 23:18:20 dist-test-slave-754n krb5kdc[13838](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 23:18:20 dist-test-slave-754n krb5kdc[13838](info): set up 2 sockets
Sep 23 23:18:20 dist-test-slave-754n krb5kdc[13838](info): commencing operation
krb5kdc: starting...
W20260923 23:18:21.415638 25487 mini_kdc.cc:121] Time spent starting KDC: real 0.974s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790205259943518-25487-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790205259943518-25487-0/krb5kdc/test-user.keytab.
Sep 23 23:18:21 dist-test-slave-754n krb5kdc[13838](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205501, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T23:18:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T23:18:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790205259943518-25487-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790205259943518-25487-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 23:18:21.523034 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790205259943518-25487-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790205259943518-25487-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790205259943518-25487-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43793
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790205259943518-25487-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43793
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:18:21.598354 13851 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:18:21.598562 13851 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:18:21.598598 13851 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:18:21.599658 13851 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 23:18:21.599725 13851 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:18:21.599749 13851 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:18:21.599771 13851 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 23:18:21.599797 13851 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 23:18:21.601208 13851 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790205259943518-25487-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790205259943518-25487-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43793
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790205259943518-25487-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43793
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13851
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790205259943518-25487-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:18:21.601478 13851 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:18:21.601686 13851 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:18:21.606114 13859 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:18:21.606151 13856 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:18:21.606115 13857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:18:21.606367 13851 server_base.cc:1034] running on GCE node
I20260923 23:18:21.606602 13851 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:18:21.606920 13851 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:18:21.608062 13851 hybrid_clock.cc:648] HybridClock initialized: now 1790205501608044 us; error 41 us; skew 500 ppm
Sep 23 23:18:21 dist-test-slave-754n krb5kdc[13838](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205501, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:18:21.610426 13851 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 23:18:21.610848 13851 webserver.cc:460] Webserver started at http://127.0.0.1:45439/ using document root <none> and password file <none>
I20260923 23:18:21.611068 13851 fs_manager.cc:359] Metadata directory not provided
I20260923 23:18:21.611125 13851 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:18:21.611263 13851 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:18:21.612280 13851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790205259943518-25487-0/minicluster-data/master-0/data/instance:
uuid: "a9231363aa1d4098b2ce84d595387c06"
format_stamp: "Formatted at 2026-09-23 23:18:21 on dist-test-slave-754n"
server_key: "54fffc1d60e33bd7805cce9133f31e16"
server_key_iv: "e4d42381249b421f4555e7abbaae7669"
server_key_version: "encryptionkey@0"
I20260923 23:18:21.612600 13851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790205259943518-25487-0/minicluster-data/master-0/wal/instance:
uuid: "a9231363aa1d4098b2ce84d595387c06"
format_stamp: "Formatted at 2026-09-23 23:18:21 on dist-test-slave-754n"
server_key: "54fffc1d60e33bd7805cce9133f31e16"
server_key_iv: "e4d42381249b421f4555e7abbaae7669"
server_key_version: "encryptionkey@0"
I20260923 23:18:21.613787 13851 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 23:18:21.614395 13866 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:18:21.614547 13851 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 23:18:21.614658 13851 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790205259943518-25487-0/minicluster-data/master-0/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790205259943518-25487-0/minicluster-data/master-0/wal
uuid: "a9231363aa1d4098b2ce84d595387c06"
format_stamp: "Formatted at 2026-09-23 23:18:21 on dist-test-slave-754n"
server_key: "54fffc1d60e33bd7805cce9133f31e16"
server_key_iv: "e4d42381249b421f4555e7abbaae7669"
server_key_version: "encryptionkey@0"
I20260923 23:18:21.614724 13851 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790205259943518-25487-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790205259943518-25487-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790205259943518-25487-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:18:21.625209 13851 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:18:21.627183 13851 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:18:21.627315 13851 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:18:21.631031 13851 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43793
I20260923 23:18:21.631069 13918 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43793 every 8 connection(s)
I20260923 23:18:21.631358 13851 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
I20260923 23:18:21.632161 13919 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:18:21.634236 13919 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9231363aa1d4098b2ce84d595387c06: Bootstrap starting.
I20260923 23:18:21.634765 13919 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a9231363aa1d4098b2ce84d595387c06: Neither blocks nor log segments found. Creating new log.
I20260923 23:18:21.635032 13919 log.cc:824] T 00000000000000000000000000000000 P a9231363aa1d4098b2ce84d595387c06: Log is configured to *not* fsync() on all Append() calls
I20260923 23:18:21.635632 13919 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9231363aa1d4098b2ce84d595387c06: No bootstrap required, opened a new log
I20260923 23:18:21.636952 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 13851
I20260923 23:18:21.637063 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790205259943518-25487-0/minicluster-data/master-0/wal/instance
I20260923 23:18:21.637301 25487 external_mini_cluster.cc:1428] Setting key 7ed5d6374ac911fdaa76e4bb19d9343c
I20260923 23:18:21.637526 13919 raft_consensus.cc:348] T 00000000000000000000000000000000 P a9231363aa1d4098b2ce84d595387c06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9231363aa1d4098b2ce84d595387c06" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43793 } }
I20260923 23:18:21.637705 13919 raft_consensus.cc:374] T 00000000000000000000000000000000 P a9231363aa1d4098b2ce84d595387c06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:18:21.637753 13919 raft_consensus.cc:729] T 00000000000000000000000000000000 P a9231363aa1d4098b2ce84d595387c06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9231363aa1d4098b2ce84d595387c06, State: Initialized, Role: FOLLOWER
I20260923 23:18:21.637914 13919 consensus_queue.cc:260] T 00000000000000000000000000000000 P a9231363aa1d4098b2ce84d595387c06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9231363aa1d4098b2ce84d595387c06" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43793 } }
I20260923 23:18:21.638007 13919 raft_consensus.cc:388] T 00000000000000000000000000000000 P a9231363aa1d4098b2ce84d595387c06 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 23:18:21.638049 13919 raft_consensus.cc:482] T 00000000000000000000000000000000 P a9231363aa1d4098b2ce84d595387c06 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 23:18:21.638098 13919 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a9231363aa1d4098b2ce84d595387c06 [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:18:21.638753 13919 raft_consensus.cc:504] T 00000000000000000000000000000000 P a9231363aa1d4098b2ce84d595387c06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9231363aa1d4098b2ce84d595387c06" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43793 } }
I20260923 23:18:21.638857 13919 leader_election.cc:302] T 00000000000000000000000000000000 P a9231363aa1d4098b2ce84d595387c06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a9231363aa1d4098b2ce84d595387c06; no voters:
I20260923 23:18:21.639009 13919 leader_election.cc:288] T 00000000000000000000000000000000 P a9231363aa1d4098b2ce84d595387c06 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 23:18:21.639046 13924 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a9231363aa1d4098b2ce84d595387c06 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 23:18:21.639209 13924 raft_consensus.cc:686] T 00000000000000000000000000000000 P a9231363aa1d4098b2ce84d595387c06 [term 1 LEADER]: Becoming Leader. State: Replica: a9231363aa1d4098b2ce84d595387c06, State: Running, Role: LEADER
I20260923 23:18:21.639230 13919 sys_catalog.cc:564] T 00000000000000000000000000000000 P a9231363aa1d4098b2ce84d595387c06 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 23:18:21.639343 13924 consensus_queue.cc:237] T 00000000000000000000000000000000 P a9231363aa1d4098b2ce84d595387c06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9231363aa1d4098b2ce84d595387c06" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43793 } }
I20260923 23:18:21.639757 13926 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9231363aa1d4098b2ce84d595387c06 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a9231363aa1d4098b2ce84d595387c06. Latest consensus state: current_term: 1 leader_uuid: "a9231363aa1d4098b2ce84d595387c06" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9231363aa1d4098b2ce84d595387c06" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43793 } } }
I20260923 23:18:21.639822 13926 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9231363aa1d4098b2ce84d595387c06 [sys.catalog]: This master's current role is: LEADER
I20260923 23:18:21.640009 13925 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9231363aa1d4098b2ce84d595387c06 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a9231363aa1d4098b2ce84d595387c06" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9231363aa1d4098b2ce84d595387c06" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43793 } } }
I20260923 23:18:21.640100 13925 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9231363aa1d4098b2ce84d595387c06 [sys.catalog]: This master's current role is: LEADER
I20260923 23:18:21.640441 13930 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 23:18:21.640936 13930 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 23:18:21.642448 13930 catalog_manager.cc:1347] Generated new cluster ID: 0c318e2cb8e444be8dcc28da90bd5fe6
I20260923 23:18:21.642503 13930 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 23:18:21 dist-test-slave-754n krb5kdc[13838](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205501, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260923 23:18:21.648465 13923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:21.637734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.172:52311 (local address 127.0.0.1:43793)
0923 23:18:21.637925 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:21.637933 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:21.637985 (+ 52us) server_negotiation.cc:407] Connection header received
0923 23:18:21.638109 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:21.638115 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:21.638179 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:21.638282 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:21.639011 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:21.640438 (+ 1427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:21.641472 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:21.642062 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:21.642172 (+ 110us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:21.644810 (+ 2638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:18:21.644814 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:18:21.644815 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:18:21.644826 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:18:21.646760 (+ 1934us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:21.647186 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:21.647189 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:21.647190 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:21.647297 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:21.647519 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:21.647520 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:21.647520 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:21.647877 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:18:21.648099 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:21.648184 (+ 85us) server_negotiation.cc:299] Negotiation successful
0923 23:18:21.648284 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":75,"threads_started":1}
I20260923 23:18:21.649345 25487 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 23:18:21.657526 13923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:21.650779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.172:57451 (local address 127.0.0.1:43793)
0923 23:18:21.650816 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:21.650817 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:21.650938 (+ 121us) server_negotiation.cc:407] Connection header received
0923 23:18:21.650981 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:21.650982 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:21.650992 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:21.651058 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:21.651573 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:21.652322 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:21.653281 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:21.653596 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:21.653659 (+ 63us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:21.654898 (+ 1239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:18:21.654899 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:18:21.654900 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:18:21.654906 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:18:21.656406 (+ 1500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:21.656799 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:21.656799 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:21.656800 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:21.656856 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:21.657071 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:21.657072 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:21.657072 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:21.657194 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:18:21.657285 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:21.657389 (+ 104us) server_negotiation.cc:299] Negotiation successful
0923 23:18:21.657445 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 23:18:21.698334 13930 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 23:18:21.699069 13930 catalog_manager.cc:1504] Loading token signing keys...
I20260923 23:18:21.704741 13930 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a9231363aa1d4098b2ce84d595387c06: Generated new TSK 0
I20260923 23:18:21.704910 13930 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 23:18:21.713546 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 13851
W20260923 23:18:21.717724 13836 connection.cc:535] client connection to 127.0.0.1:43793 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43793): Connection reset by peer (error 104)
2026-09-23T23:18:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1289 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790205259943518-25487-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 23:18:21 dist-test-slave-754n krb5kdc[13950](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 23:18:21 dist-test-slave-754n krb5kdc[13950](info): set up 2 sockets
Sep 23 23:18:21 dist-test-slave-754n krb5kdc[13950](info): commencing operation
krb5kdc: starting...
W20260923 23:18:22.711607 25487 mini_kdc.cc:121] Time spent starting KDC: real 0.980s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790205259943518-25487-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790205259943518-25487-0/krb5kdc/test-user.keytab.
Sep 23 23:18:22 dist-test-slave-754n krb5kdc[13950](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205502, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T23:18:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T23:18:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790205259943518-25487-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790205259943518-25487-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 23:18:22.816901 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790205259943518-25487-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790205259943518-25487-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790205259943518-25487-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34531
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790205259943518-25487-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34531
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:18:22.893705 13963 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:18:22.893901 13963 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:18:22.893936 13963 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:18:22.894989 13963 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 23:18:22.895045 13963 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:18:22.895068 13963 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:18:22.895087 13963 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 23:18:22.895113 13963 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 23:18:22.896443 13963 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790205259943518-25487-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790205259943518-25487-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34531
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790205259943518-25487-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34531
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13963
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790205259943518-25487-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:18:22.896688 13963 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:18:22.896876 13963 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:18:22.901160 13971 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:18:22.901171 13969 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:18:22.901293 13963 server_base.cc:1034] running on GCE node
W20260923 23:18:22.901201 13968 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:18:22.901599 13963 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:18:22.901878 13963 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:18:22.903012 13963 hybrid_clock.cc:648] HybridClock initialized: now 1790205502903001 us; error 30 us; skew 500 ppm
Sep 23 23:18:22 dist-test-slave-754n krb5kdc[13950](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205502, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:18:22.905169 13963 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 23:18:22.905517 13963 webserver.cc:460] Webserver started at http://127.0.0.1:39781/ using document root <none> and password file <none>
I20260923 23:18:22.905706 13963 fs_manager.cc:359] Metadata directory not provided
I20260923 23:18:22.905763 13963 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:18:22.905894 13963 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:18:22.906852 13963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790205259943518-25487-0/minicluster-data/master-0/data/instance:
uuid: "7d538dc78ad0407db8ff0835592c86a2"
format_stamp: "Formatted at 2026-09-23 23:18:22 on dist-test-slave-754n"
server_key: "7921861d02cf8eda6b7441f2417187a3"
server_key_iv: "1b17a770041c75d8f9dafc50096a8dff"
server_key_version: "encryptionkey@0"
I20260923 23:18:22.907176 13963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790205259943518-25487-0/minicluster-data/master-0/wal/instance:
uuid: "7d538dc78ad0407db8ff0835592c86a2"
format_stamp: "Formatted at 2026-09-23 23:18:22 on dist-test-slave-754n"
server_key: "7921861d02cf8eda6b7441f2417187a3"
server_key_iv: "1b17a770041c75d8f9dafc50096a8dff"
server_key_version: "encryptionkey@0"
I20260923 23:18:22.908384 13963 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260923 23:18:22.908990 13978 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:18:22.909127 13963 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 23:18:22.909215 13963 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790205259943518-25487-0/minicluster-data/master-0/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790205259943518-25487-0/minicluster-data/master-0/wal
uuid: "7d538dc78ad0407db8ff0835592c86a2"
format_stamp: "Formatted at 2026-09-23 23:18:22 on dist-test-slave-754n"
server_key: "7921861d02cf8eda6b7441f2417187a3"
server_key_iv: "1b17a770041c75d8f9dafc50096a8dff"
server_key_version: "encryptionkey@0"
I20260923 23:18:22.909281 13963 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790205259943518-25487-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790205259943518-25487-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790205259943518-25487-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:18:22.926146 13963 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:18:22.927734 13963 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:18:22.927917 13963 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:18:22.931475 13963 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34531
I20260923 23:18:22.931509 14030 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34531 every 8 connection(s)
I20260923 23:18:22.931846 13963 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
I20260923 23:18:22.932662 14031 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:18:22.934705 14031 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d538dc78ad0407db8ff0835592c86a2: Bootstrap starting.
I20260923 23:18:22.935280 14031 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7d538dc78ad0407db8ff0835592c86a2: Neither blocks nor log segments found. Creating new log.
I20260923 23:18:22.935556 14031 log.cc:824] T 00000000000000000000000000000000 P 7d538dc78ad0407db8ff0835592c86a2: Log is configured to *not* fsync() on all Append() calls
I20260923 23:18:22.936244 14031 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d538dc78ad0407db8ff0835592c86a2: No bootstrap required, opened a new log
I20260923 23:18:22.937680 14031 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7d538dc78ad0407db8ff0835592c86a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d538dc78ad0407db8ff0835592c86a2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34531 } }
I20260923 23:18:22.937803 14031 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7d538dc78ad0407db8ff0835592c86a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:18:22.937855 14031 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7d538dc78ad0407db8ff0835592c86a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d538dc78ad0407db8ff0835592c86a2, State: Initialized, Role: FOLLOWER
I20260923 23:18:22.938014 14031 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7d538dc78ad0407db8ff0835592c86a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d538dc78ad0407db8ff0835592c86a2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34531 } }
I20260923 23:18:22.938088 14031 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7d538dc78ad0407db8ff0835592c86a2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 23:18:22.938140 14031 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7d538dc78ad0407db8ff0835592c86a2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 23:18:22.938190 14031 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7d538dc78ad0407db8ff0835592c86a2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:18:22.938704 14031 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7d538dc78ad0407db8ff0835592c86a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d538dc78ad0407db8ff0835592c86a2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34531 } }
I20260923 23:18:22.938805 14031 leader_election.cc:302] T 00000000000000000000000000000000 P 7d538dc78ad0407db8ff0835592c86a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7d538dc78ad0407db8ff0835592c86a2; no voters:
I20260923 23:18:22.938972 14031 leader_election.cc:288] T 00000000000000000000000000000000 P 7d538dc78ad0407db8ff0835592c86a2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 23:18:22.939016 14034 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7d538dc78ad0407db8ff0835592c86a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 23:18:22.939172 14034 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7d538dc78ad0407db8ff0835592c86a2 [term 1 LEADER]: Becoming Leader. State: Replica: 7d538dc78ad0407db8ff0835592c86a2, State: Running, Role: LEADER
I20260923 23:18:22.939222 14031 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7d538dc78ad0407db8ff0835592c86a2 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 23:18:22.939293 14034 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7d538dc78ad0407db8ff0835592c86a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d538dc78ad0407db8ff0835592c86a2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34531 } }
I20260923 23:18:22.939615 14035 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d538dc78ad0407db8ff0835592c86a2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7d538dc78ad0407db8ff0835592c86a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d538dc78ad0407db8ff0835592c86a2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34531 } } }
I20260923 23:18:22.939618 14036 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d538dc78ad0407db8ff0835592c86a2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7d538dc78ad0407db8ff0835592c86a2. Latest consensus state: current_term: 1 leader_uuid: "7d538dc78ad0407db8ff0835592c86a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d538dc78ad0407db8ff0835592c86a2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34531 } } }
I20260923 23:18:22.939736 14035 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d538dc78ad0407db8ff0835592c86a2 [sys.catalog]: This master's current role is: LEADER
I20260923 23:18:22.939797 14036 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d538dc78ad0407db8ff0835592c86a2 [sys.catalog]: This master's current role is: LEADER
I20260923 23:18:22.940780 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 13963
I20260923 23:18:22.940893 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790205259943518-25487-0/minicluster-data/master-0/wal/instance
I20260923 23:18:22.941036 25487 external_mini_cluster.cc:1428] Setting key 530bac3728e5a4f0415e6bd86b5bad89
W20260923 23:18:22.941143 14050 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7d538dc78ad0407db8ff0835592c86a2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 23:18:22.941359 14050 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 23:18:22.941453 14038 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 23:18:22.941593 14038 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 23:18:22.942898 14038 catalog_manager.cc:1347] Generated new cluster ID: 64a33f89ae87410295c71abfb1dd3ff0
I20260923 23:18:22.942945 14038 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 23:18:22.947247 14053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:22.941573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.172:42057 (local address 127.0.0.1:34531)
0923 23:18:22.941742 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:22.941750 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:22.941795 (+ 45us) server_negotiation.cc:407] Connection header received
0923 23:18:22.942057 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:22.942065 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:22.942121 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:22.942260 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:22.942631 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:22.943993 (+ 1362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:22.945044 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:22.945613 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:22.945707 (+ 94us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:22.946387 (+ 680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:18:22.946390 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:18:22.946391 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 23:18:22.946568 (+ 177us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:18:22.946599 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 23:18:22.946611 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:18:22.946684 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:22.946895 (+ 211us) server_negotiation.cc:299] Negotiation successful
0923 23:18:22.946919 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":125,"thread_start_us":82,"threads_started":1}
I20260923 23:18:22.948087 25487 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 23:18:22.952809 14053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:22.949628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.172:45155 (local address 127.0.0.1:34531)
0923 23:18:22.949664 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:22.949665 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:22.949675 (+ 10us) server_negotiation.cc:407] Connection header received
0923 23:18:22.949746 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:22.949747 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:22.949765 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:22.949813 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:22.949942 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:22.950624 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:22.951611 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:22.951928 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:22.951977 (+ 49us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:22.952599 (+ 622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:18:22.952599 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:18:22.952600 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 23:18:22.952603 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:18:22.952613 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 23:18:22.952616 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:18:22.952669 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:22.952738 (+ 69us) server_negotiation.cc:299] Negotiation successful
0923 23:18:22.952747 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260923 23:18:22.977253 14038 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 23:18:22.977767 14038 catalog_manager.cc:1504] Loading token signing keys...
I20260923 23:18:22.985116 14038 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7d538dc78ad0407db8ff0835592c86a2: Generated new TSK 0
I20260923 23:18:22.985270 14038 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 23:18:22.990440 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 13963
W20260923 23:18:22.994771 13948 connection.cc:535] client connection to 127.0.0.1:34531 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34531): Connection reset by peer (error 104)
2026-09-23T23:18:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1277 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790205259943518-25487-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 23:18:23 dist-test-slave-754n krb5kdc[14062](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 23:18:23 dist-test-slave-754n krb5kdc[14062](info): set up 2 sockets
Sep 23 23:18:23 dist-test-slave-754n krb5kdc[14062](info): commencing operation
krb5kdc: starting...
W20260923 23:18:23.975715 25487 mini_kdc.cc:121] Time spent starting KDC: real 0.967s user 0.005s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790205259943518-25487-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790205259943518-25487-0/krb5kdc/test-user.keytab.
Sep 23 23:18:24 dist-test-slave-754n krb5kdc[14062](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205504, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T23:18:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T23:18:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790205259943518-25487-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790205259943518-25487-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 23:18:24.082646 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790205259943518-25487-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790205259943518-25487-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790205259943518-25487-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46199
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790205259943518-25487-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46199
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:18:24.158578 14075 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:18:24.158785 14075 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:18:24.158819 14075 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:18:24.159961 14075 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 23:18:24.160022 14075 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:18:24.160044 14075 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:18:24.160095 14075 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 23:18:24.160126 14075 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 23:18:24.161543 14075 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790205259943518-25487-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790205259943518-25487-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46199
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790205259943518-25487-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46199
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14075
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790205259943518-25487-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:18:24.161803 14075 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:18:24.162005 14075 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:18:24.166666 14083 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:18:24.166626 14075 server_base.cc:1034] running on GCE node
W20260923 23:18:24.166564 14080 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:18:24.166591 14081 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:18:24.167035 14075 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:18:24.167289 14075 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:18:24.168435 14075 hybrid_clock.cc:648] HybridClock initialized: now 1790205504168424 us; error 34 us; skew 500 ppm
Sep 23 23:18:24 dist-test-slave-754n krb5kdc[14062](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205504, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:18:24.170940 14075 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 23:18:24.171363 14075 webserver.cc:460] Webserver started at http://127.0.0.1:44841/ using document root <none> and password file <none>
I20260923 23:18:24.171530 14075 fs_manager.cc:359] Metadata directory not provided
I20260923 23:18:24.171567 14075 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:18:24.171655 14075 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:18:24.172652 14075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790205259943518-25487-0/minicluster-data/master-0/data/instance:
uuid: "c7ce62f0c85f4455aab203a7e714fac1"
format_stamp: "Formatted at 2026-09-23 23:18:24 on dist-test-slave-754n"
server_key: "fb666f044ab66b071b37bda2450cfd45"
server_key_iv: "1454b2fec45cac0d233a9bc9fdaad86b"
server_key_version: "encryptionkey@0"
I20260923 23:18:24.172943 14075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790205259943518-25487-0/minicluster-data/master-0/wal/instance:
uuid: "c7ce62f0c85f4455aab203a7e714fac1"
format_stamp: "Formatted at 2026-09-23 23:18:24 on dist-test-slave-754n"
server_key: "fb666f044ab66b071b37bda2450cfd45"
server_key_iv: "1454b2fec45cac0d233a9bc9fdaad86b"
server_key_version: "encryptionkey@0"
I20260923 23:18:24.174160 14075 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260923 23:18:24.174827 14090 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:18:24.174973 14075 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 23:18:24.175052 14075 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790205259943518-25487-0/minicluster-data/master-0/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790205259943518-25487-0/minicluster-data/master-0/wal
uuid: "c7ce62f0c85f4455aab203a7e714fac1"
format_stamp: "Formatted at 2026-09-23 23:18:24 on dist-test-slave-754n"
server_key: "fb666f044ab66b071b37bda2450cfd45"
server_key_iv: "1454b2fec45cac0d233a9bc9fdaad86b"
server_key_version: "encryptionkey@0"
I20260923 23:18:24.175107 14075 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790205259943518-25487-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790205259943518-25487-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790205259943518-25487-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:18:24.178202 14075 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:18:24.179937 14075 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:18:24.180042 14075 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:18:24.183561 14075 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46199
I20260923 23:18:24.183605 14142 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46199 every 8 connection(s)
I20260923 23:18:24.183964 14075 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
I20260923 23:18:24.184865 14143 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:18:24.186587 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 14075
I20260923 23:18:24.186717 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790205259943518-25487-0/minicluster-data/master-0/wal/instance
I20260923 23:18:24.186897 25487 external_mini_cluster.cc:1428] Setting key d14c452e609c412d311d97886f26d76f
I20260923 23:18:24.187059 14143 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7ce62f0c85f4455aab203a7e714fac1: Bootstrap starting.
I20260923 23:18:24.187731 14143 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c7ce62f0c85f4455aab203a7e714fac1: Neither blocks nor log segments found. Creating new log.
I20260923 23:18:24.188055 14143 log.cc:824] T 00000000000000000000000000000000 P c7ce62f0c85f4455aab203a7e714fac1: Log is configured to *not* fsync() on all Append() calls
I20260923 23:18:24.188745 14143 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7ce62f0c85f4455aab203a7e714fac1: No bootstrap required, opened a new log
I20260923 23:18:24.189610 14146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:24.187553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.172:54867 (local address 127.0.0.1:46199)
0923 23:18:24.187738 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:24.187747 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:24.187777 (+ 30us) server_negotiation.cc:407] Connection header received
0923 23:18:24.188371 (+ 594us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:24.188378 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:24.188404 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:24.188561 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:24.189154 (+ 593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:18:24.189157 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:18:24.189158 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 23:18:24.189167 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:18:24.189209 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 23:18:24.189224 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:18:24.189316 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:24.189355 (+ 39us) server_negotiation.cc:299] Negotiation successful
0923 23:18:24.189392 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":71,"threads_started":1}
I20260923 23:18:24.190459 25487 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 23:18:24.191097 14143 raft_consensus.cc:348] T 00000000000000000000000000000000 P c7ce62f0c85f4455aab203a7e714fac1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7ce62f0c85f4455aab203a7e714fac1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46199 } }
I20260923 23:18:24.191214 14143 raft_consensus.cc:374] T 00000000000000000000000000000000 P c7ce62f0c85f4455aab203a7e714fac1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:18:24.191236 14143 raft_consensus.cc:729] T 00000000000000000000000000000000 P c7ce62f0c85f4455aab203a7e714fac1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7ce62f0c85f4455aab203a7e714fac1, State: Initialized, Role: FOLLOWER
I20260923 23:18:24.191370 14143 consensus_queue.cc:260] T 00000000000000000000000000000000 P c7ce62f0c85f4455aab203a7e714fac1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7ce62f0c85f4455aab203a7e714fac1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46199 } }
I20260923 23:18:24.191473 14143 raft_consensus.cc:388] T 00000000000000000000000000000000 P c7ce62f0c85f4455aab203a7e714fac1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 23:18:24.191505 14143 raft_consensus.cc:482] T 00000000000000000000000000000000 P c7ce62f0c85f4455aab203a7e714fac1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 23:18:24.191550 14143 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c7ce62f0c85f4455aab203a7e714fac1 [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:18:24.192322 14143 raft_consensus.cc:504] T 00000000000000000000000000000000 P c7ce62f0c85f4455aab203a7e714fac1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7ce62f0c85f4455aab203a7e714fac1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46199 } }
I20260923 23:18:24.192431 14143 leader_election.cc:302] T 00000000000000000000000000000000 P c7ce62f0c85f4455aab203a7e714fac1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c7ce62f0c85f4455aab203a7e714fac1; no voters:
I20260923 23:18:24.192595 14143 leader_election.cc:288] T 00000000000000000000000000000000 P c7ce62f0c85f4455aab203a7e714fac1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 23:18:24.192713 14146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:24.191722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.172:39957 (local address 127.0.0.1:46199)
0923 23:18:24.191755 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:24.191757 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:24.191915 (+ 158us) server_negotiation.cc:407] Connection header received
0923 23:18:24.191974 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:24.191974 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:24.191978 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:24.192018 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:24.192511 (+ 493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:18:24.192512 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:18:24.192513 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 23:18:24.192517 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:18:24.192530 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 23:18:24.192534 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:18:24.192578 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:24.192639 (+ 61us) server_negotiation.cc:299] Negotiation successful
0923 23:18:24.192652 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260923 23:18:24.192883 14143 sys_catalog.cc:564] T 00000000000000000000000000000000 P c7ce62f0c85f4455aab203a7e714fac1 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 23:18:24.192613 14153 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c7ce62f0c85f4455aab203a7e714fac1 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 23:18:24.194118 14153 raft_consensus.cc:686] T 00000000000000000000000000000000 P c7ce62f0c85f4455aab203a7e714fac1 [term 1 LEADER]: Becoming Leader. State: Replica: c7ce62f0c85f4455aab203a7e714fac1, State: Running, Role: LEADER
I20260923 23:18:24.194214 14153 consensus_queue.cc:237] T 00000000000000000000000000000000 P c7ce62f0c85f4455aab203a7e714fac1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7ce62f0c85f4455aab203a7e714fac1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46199 } }
I20260923 23:18:24.194509 14153 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7ce62f0c85f4455aab203a7e714fac1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c7ce62f0c85f4455aab203a7e714fac1. Latest consensus state: current_term: 1 leader_uuid: "c7ce62f0c85f4455aab203a7e714fac1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7ce62f0c85f4455aab203a7e714fac1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46199 } } }
I20260923 23:18:24.194522 14165 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7ce62f0c85f4455aab203a7e714fac1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c7ce62f0c85f4455aab203a7e714fac1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7ce62f0c85f4455aab203a7e714fac1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46199 } } }
I20260923 23:18:24.194634 14153 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7ce62f0c85f4455aab203a7e714fac1 [sys.catalog]: This master's current role is: LEADER
I20260923 23:18:24.194700 14165 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7ce62f0c85f4455aab203a7e714fac1 [sys.catalog]: This master's current role is: LEADER
W20260923 23:18:24.195158 14167 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c7ce62f0c85f4455aab203a7e714fac1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 23:18:24.195214 14167 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 23:18:24.195298 14169 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 23:18:24.195490 14169 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 23:18:24.196753 14169 catalog_manager.cc:1347] Generated new cluster ID: 71647fe60171411b8c3ae6c411001519
I20260923 23:18:24.196789 14169 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 23:18:24.207399 14169 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 23:18:24.207525 14169 catalog_manager.cc:1504] Loading token signing keys...
I20260923 23:18:24.215277 14169 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c7ce62f0c85f4455aab203a7e714fac1: Generated new TSK 0
I20260923 23:18:24.215436 14169 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 23:18:24.222551 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 14075
2026-09-23T23:18:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1231 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790205259943518-25487-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 23:18:24 dist-test-slave-754n krb5kdc[14174](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 23:18:24 dist-test-slave-754n krb5kdc[14174](info): set up 2 sockets
Sep 23 23:18:24 dist-test-slave-754n krb5kdc[14174](info): commencing operation
krb5kdc: starting...
W20260923 23:18:25.211846 25487 mini_kdc.cc:121] Time spent starting KDC: real 0.972s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790205259943518-25487-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790205259943518-25487-0/krb5kdc/test-user.keytab.
Sep 23 23:18:25 dist-test-slave-754n krb5kdc[14174](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205505, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T23:18:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T23:18:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790205259943518-25487-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790205259943518-25487-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 23:18:25.318106 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790205259943518-25487-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790205259943518-25487-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790205259943518-25487-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42773
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790205259943518-25487-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42773
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:18:25.398159 14187 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:18:25.398381 14187 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:18:25.398422 14187 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:18:25.399555 14187 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 23:18:25.399619 14187 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:18:25.399643 14187 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:18:25.399672 14187 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 23:18:25.399695 14187 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 23:18:25.401140 14187 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790205259943518-25487-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790205259943518-25487-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42773
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790205259943518-25487-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42773
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14187
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790205259943518-25487-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:18:25.401402 14187 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:18:25.401661 14187 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:18:25.406279 14193 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:18:25.406324 14187 server_base.cc:1034] running on GCE node
W20260923 23:18:25.406270 14195 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:18:25.406286 14192 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:18:25.406733 14187 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:18:25.406952 14187 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:18:25.408092 14187 hybrid_clock.cc:648] HybridClock initialized: now 1790205505408072 us; error 34 us; skew 500 ppm
Sep 23 23:18:25 dist-test-slave-754n krb5kdc[14174](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205505, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:18:25.413580 14187 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 23:18:25.414008 14187 webserver.cc:460] Webserver started at http://127.0.0.1:36021/ using document root <none> and password file <none>
I20260923 23:18:25.414222 14187 fs_manager.cc:359] Metadata directory not provided
I20260923 23:18:25.414286 14187 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:18:25.414433 14187 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:18:25.415388 14187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790205259943518-25487-0/minicluster-data/master-0/data/instance:
uuid: "eaa8b48a6bfb4e5a8e6a174c90ba25c5"
format_stamp: "Formatted at 2026-09-23 23:18:25 on dist-test-slave-754n"
server_key: "21ad9c9eeecb177ec8e6e893b045eca5"
server_key_iv: "aed25b8711da4bb86f0006b6bf9e43a5"
server_key_version: "encryptionkey@0"
I20260923 23:18:25.415727 14187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790205259943518-25487-0/minicluster-data/master-0/wal/instance:
uuid: "eaa8b48a6bfb4e5a8e6a174c90ba25c5"
format_stamp: "Formatted at 2026-09-23 23:18:25 on dist-test-slave-754n"
server_key: "21ad9c9eeecb177ec8e6e893b045eca5"
server_key_iv: "aed25b8711da4bb86f0006b6bf9e43a5"
server_key_version: "encryptionkey@0"
I20260923 23:18:25.416997 14187 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 23:18:25.417654 14202 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:18:25.417819 14187 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 23:18:25.417913 14187 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790205259943518-25487-0/minicluster-data/master-0/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790205259943518-25487-0/minicluster-data/master-0/wal
uuid: "eaa8b48a6bfb4e5a8e6a174c90ba25c5"
format_stamp: "Formatted at 2026-09-23 23:18:25 on dist-test-slave-754n"
server_key: "21ad9c9eeecb177ec8e6e893b045eca5"
server_key_iv: "aed25b8711da4bb86f0006b6bf9e43a5"
server_key_version: "encryptionkey@0"
I20260923 23:18:25.417991 14187 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790205259943518-25487-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790205259943518-25487-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790205259943518-25487-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:18:25.444933 14187 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:18:25.446565 14187 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:18:25.446712 14187 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:18:25.450927 14187 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42773
I20260923 23:18:25.450954 14254 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42773 every 8 connection(s)
I20260923 23:18:25.451265 14187 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
I20260923 23:18:25.452104 14255 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:18:25.452247 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 14187
I20260923 23:18:25.452381 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790205259943518-25487-0/minicluster-data/master-0/wal/instance
I20260923 23:18:25.452589 25487 external_mini_cluster.cc:1428] Setting key 0b87b6b4c4e13d54e2ccc2b99a6fc68f
I20260923 23:18:25.454833 14255 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eaa8b48a6bfb4e5a8e6a174c90ba25c5: Bootstrap starting.
I20260923 23:18:25.455389 14255 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eaa8b48a6bfb4e5a8e6a174c90ba25c5: Neither blocks nor log segments found. Creating new log.
I20260923 23:18:25.455684 14255 log.cc:824] T 00000000000000000000000000000000 P eaa8b48a6bfb4e5a8e6a174c90ba25c5: Log is configured to *not* fsync() on all Append() calls
I20260923 23:18:25.456445 14255 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eaa8b48a6bfb4e5a8e6a174c90ba25c5: No bootstrap required, opened a new log
I20260923 23:18:25.458864 14255 raft_consensus.cc:348] T 00000000000000000000000000000000 P eaa8b48a6bfb4e5a8e6a174c90ba25c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eaa8b48a6bfb4e5a8e6a174c90ba25c5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42773 } }
I20260923 23:18:25.458988 14255 raft_consensus.cc:374] T 00000000000000000000000000000000 P eaa8b48a6bfb4e5a8e6a174c90ba25c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:18:25.459009 14255 raft_consensus.cc:729] T 00000000000000000000000000000000 P eaa8b48a6bfb4e5a8e6a174c90ba25c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eaa8b48a6bfb4e5a8e6a174c90ba25c5, State: Initialized, Role: FOLLOWER
I20260923 23:18:25.459148 14255 consensus_queue.cc:260] T 00000000000000000000000000000000 P eaa8b48a6bfb4e5a8e6a174c90ba25c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eaa8b48a6bfb4e5a8e6a174c90ba25c5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42773 } }
I20260923 23:18:25.459209 14255 raft_consensus.cc:388] T 00000000000000000000000000000000 P eaa8b48a6bfb4e5a8e6a174c90ba25c5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
Sep 23 23:18:25 dist-test-slave-754n krb5kdc[14174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205505, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260923 23:18:25.459229 14255 raft_consensus.cc:482] T 00000000000000000000000000000000 P eaa8b48a6bfb4e5a8e6a174c90ba25c5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 23:18:25.459295 14255 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eaa8b48a6bfb4e5a8e6a174c90ba25c5 [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:18:25.459831 14255 raft_consensus.cc:504] T 00000000000000000000000000000000 P eaa8b48a6bfb4e5a8e6a174c90ba25c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eaa8b48a6bfb4e5a8e6a174c90ba25c5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42773 } }
I20260923 23:18:25.459977 14255 leader_election.cc:302] T 00000000000000000000000000000000 P eaa8b48a6bfb4e5a8e6a174c90ba25c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eaa8b48a6bfb4e5a8e6a174c90ba25c5; no voters:
I20260923 23:18:25.460153 14255 leader_election.cc:288] T 00000000000000000000000000000000 P eaa8b48a6bfb4e5a8e6a174c90ba25c5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 23:18:25.460307 14260 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eaa8b48a6bfb4e5a8e6a174c90ba25c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 23:18:25.460383 14255 sys_catalog.cc:564] T 00000000000000000000000000000000 P eaa8b48a6bfb4e5a8e6a174c90ba25c5 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 23:18:25.460458 14260 raft_consensus.cc:686] T 00000000000000000000000000000000 P eaa8b48a6bfb4e5a8e6a174c90ba25c5 [term 1 LEADER]: Becoming Leader. State: Replica: eaa8b48a6bfb4e5a8e6a174c90ba25c5, State: Running, Role: LEADER
I20260923 23:18:25.460572 14260 consensus_queue.cc:237] T 00000000000000000000000000000000 P eaa8b48a6bfb4e5a8e6a174c90ba25c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eaa8b48a6bfb4e5a8e6a174c90ba25c5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42773 } }
I20260923 23:18:25.461102 14261 sys_catalog.cc:455] T 00000000000000000000000000000000 P eaa8b48a6bfb4e5a8e6a174c90ba25c5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eaa8b48a6bfb4e5a8e6a174c90ba25c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eaa8b48a6bfb4e5a8e6a174c90ba25c5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42773 } } }
I20260923 23:18:25.461113 14262 sys_catalog.cc:455] T 00000000000000000000000000000000 P eaa8b48a6bfb4e5a8e6a174c90ba25c5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eaa8b48a6bfb4e5a8e6a174c90ba25c5. Latest consensus state: current_term: 1 leader_uuid: "eaa8b48a6bfb4e5a8e6a174c90ba25c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eaa8b48a6bfb4e5a8e6a174c90ba25c5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42773 } } }
I20260923 23:18:25.461234 14262 sys_catalog.cc:458] T 00000000000000000000000000000000 P eaa8b48a6bfb4e5a8e6a174c90ba25c5 [sys.catalog]: This master's current role is: LEADER
I20260923 23:18:25.461192 14261 sys_catalog.cc:458] T 00000000000000000000000000000000 P eaa8b48a6bfb4e5a8e6a174c90ba25c5 [sys.catalog]: This master's current role is: LEADER
I20260923 23:18:25.461485 14267 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 23:18:25.461993 14267 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 23:18:25.463881 14267 catalog_manager.cc:1347] Generated new cluster ID: 41ba4e25c6754099ab88928be4cd89bf
I20260923 23:18:25.463902 14258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:25.453079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.172:59875 (local address 127.0.0.1:42773)
0923 23:18:25.453275 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:25.453284 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:25.453317 (+ 33us) server_negotiation.cc:407] Connection header received
0923 23:18:25.453689 (+ 372us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:25.453694 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:25.453736 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:25.453843 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:25.454574 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:25.455488 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:25.456532 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:25.456911 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:25.456999 (+ 88us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:25.459739 (+ 2740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:18:25.459743 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:18:25.459743 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:18:25.459766 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:18:25.462107 (+ 2341us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:25.462582 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:25.462585 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:25.462585 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:25.462705 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:25.462984 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:25.462985 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:25.462986 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:25.463280 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:18:25.463434 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:25.463596 (+ 162us) server_negotiation.cc:299] Negotiation successful
0923 23:18:25.463688 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":104,"threads_started":1}
I20260923 23:18:25.463932 14267 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 23:18:25.464728 25487 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 23:18:25.472858 14267 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 23:18:25.473330 14267 catalog_manager.cc:1504] Loading token signing keys...
I20260923 23:18:25.478561 14258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:25.468250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.172:39705 (local address 127.0.0.1:42773)
0923 23:18:25.468283 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:25.468285 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:25.472049 (+ 3764us) server_negotiation.cc:407] Connection header received
0923 23:18:25.472121 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:25.472122 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:25.472131 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:25.472177 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:25.472704 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:25.473421 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:25.474374 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:25.474686 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:25.474754 (+ 68us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:25.475966 (+ 1212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:18:25.475967 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:18:25.475968 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:18:25.475974 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:18:25.477410 (+ 1436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:25.477762 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:25.477763 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:25.477764 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:25.477816 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:25.478080 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:25.478081 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:25.478081 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:25.478195 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:18:25.478288 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:25.478421 (+ 133us) server_negotiation.cc:299] Negotiation successful
0923 23:18:25.478481 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260923 23:18:25.483007 14267 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eaa8b48a6bfb4e5a8e6a174c90ba25c5: Generated new TSK 0
I20260923 23:18:25.483184 14267 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 23:18:25.487668 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 14187
W20260923 23:18:25.492379 14172 connection.cc:535] client connection to 127.0.0.1:42773 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42773): Connection reset by peer (error 104)
2026-09-23T23:18:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1265 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790205259943518-25487-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 23:18:25 dist-test-slave-754n krb5kdc[14286](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 23:18:25 dist-test-slave-754n krb5kdc[14286](info): set up 2 sockets
Sep 23 23:18:25 dist-test-slave-754n krb5kdc[14286](info): commencing operation
krb5kdc: starting...
W20260923 23:18:26.471915 25487 mini_kdc.cc:121] Time spent starting KDC: real 0.966s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790205259943518-25487-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790205259943518-25487-0/krb5kdc/test-user.keytab.
Sep 23 23:18:26 dist-test-slave-754n krb5kdc[14286](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205506, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T23:18:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T23:18:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790205259943518-25487-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790205259943518-25487-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 23:18:26.578039 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790205259943518-25487-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790205259943518-25487-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790205259943518-25487-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45695
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790205259943518-25487-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45695
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:18:26.654948 14299 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:18:26.655124 14299 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:18:26.655144 14299 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:18:26.656178 14299 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 23:18:26.656214 14299 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:18:26.656227 14299 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:18:26.656239 14299 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 23:18:26.656270 14299 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 23:18:26.657577 14299 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790205259943518-25487-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790205259943518-25487-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45695
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790205259943518-25487-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45695
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14299
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790205259943518-25487-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:18:26.657752 14299 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:18:26.657968 14299 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260923 23:18:26.662580 14299 server_base.cc:1034] running on GCE node
W20260923 23:18:26.662554 14307 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:18:26.662555 14305 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:18:26.662572 14304 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:18:26.662959 14299 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:18:26.663229 14299 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:18:26.664382 14299 hybrid_clock.cc:648] HybridClock initialized: now 1790205506664372 us; error 35 us; skew 500 ppm
Sep 23 23:18:26 dist-test-slave-754n krb5kdc[14286](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205506, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:18:26.666442 14299 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 23:18:26.666802 14299 webserver.cc:460] Webserver started at http://127.0.0.1:39165/ using document root <none> and password file <none>
I20260923 23:18:26.666991 14299 fs_manager.cc:359] Metadata directory not provided
I20260923 23:18:26.667057 14299 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:18:26.667191 14299 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:18:26.668135 14299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790205259943518-25487-0/minicluster-data/master-0/data/instance:
uuid: "325bd09ed3ba4aeaad5e2ebc7947f96c"
format_stamp: "Formatted at 2026-09-23 23:18:26 on dist-test-slave-754n"
server_key: "a642b44a04c9b5a4de29da3fd8a988d2"
server_key_iv: "16316d43a5c718a1e7ad4bae28c8377e"
server_key_version: "encryptionkey@0"
I20260923 23:18:26.668449 14299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790205259943518-25487-0/minicluster-data/master-0/wal/instance:
uuid: "325bd09ed3ba4aeaad5e2ebc7947f96c"
format_stamp: "Formatted at 2026-09-23 23:18:26 on dist-test-slave-754n"
server_key: "a642b44a04c9b5a4de29da3fd8a988d2"
server_key_iv: "16316d43a5c718a1e7ad4bae28c8377e"
server_key_version: "encryptionkey@0"
I20260923 23:18:26.669732 14299 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 23:18:26.670425 14314 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:18:26.670585 14299 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 23:18:26.670689 14299 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790205259943518-25487-0/minicluster-data/master-0/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790205259943518-25487-0/minicluster-data/master-0/wal
uuid: "325bd09ed3ba4aeaad5e2ebc7947f96c"
format_stamp: "Formatted at 2026-09-23 23:18:26 on dist-test-slave-754n"
server_key: "a642b44a04c9b5a4de29da3fd8a988d2"
server_key_iv: "16316d43a5c718a1e7ad4bae28c8377e"
server_key_version: "encryptionkey@0"
I20260923 23:18:26.670766 14299 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790205259943518-25487-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790205259943518-25487-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790205259943518-25487-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:18:26.680233 14299 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:18:26.681891 14299 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:18:26.682072 14299 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:18:26.686420 14299 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45695
I20260923 23:18:26.686453 14366 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45695 every 8 connection(s)
I20260923 23:18:26.686802 14299 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
I20260923 23:18:26.687476 14367 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:18:26.689383 14367 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 325bd09ed3ba4aeaad5e2ebc7947f96c: Bootstrap starting.
I20260923 23:18:26.689888 14367 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 325bd09ed3ba4aeaad5e2ebc7947f96c: Neither blocks nor log segments found. Creating new log.
I20260923 23:18:26.690120 14367 log.cc:824] T 00000000000000000000000000000000 P 325bd09ed3ba4aeaad5e2ebc7947f96c: Log is configured to *not* fsync() on all Append() calls
I20260923 23:18:26.690706 14367 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 325bd09ed3ba4aeaad5e2ebc7947f96c: No bootstrap required, opened a new log
I20260923 23:18:26.691778 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 14299
I20260923 23:18:26.691932 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790205259943518-25487-0/minicluster-data/master-0/wal/instance
I20260923 23:18:26.692142 25487 external_mini_cluster.cc:1428] Setting key 8c689e602ee39f8ef403f015f283a2f8
I20260923 23:18:26.692417 14367 raft_consensus.cc:348] T 00000000000000000000000000000000 P 325bd09ed3ba4aeaad5e2ebc7947f96c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "325bd09ed3ba4aeaad5e2ebc7947f96c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45695 } }
I20260923 23:18:26.692624 14367 raft_consensus.cc:374] T 00000000000000000000000000000000 P 325bd09ed3ba4aeaad5e2ebc7947f96c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:18:26.692678 14367 raft_consensus.cc:729] T 00000000000000000000000000000000 P 325bd09ed3ba4aeaad5e2ebc7947f96c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 325bd09ed3ba4aeaad5e2ebc7947f96c, State: Initialized, Role: FOLLOWER
I20260923 23:18:26.692860 14367 consensus_queue.cc:260] T 00000000000000000000000000000000 P 325bd09ed3ba4aeaad5e2ebc7947f96c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "325bd09ed3ba4aeaad5e2ebc7947f96c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45695 } }
I20260923 23:18:26.692950 14367 raft_consensus.cc:388] T 00000000000000000000000000000000 P 325bd09ed3ba4aeaad5e2ebc7947f96c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 23:18:26.692978 14367 raft_consensus.cc:482] T 00000000000000000000000000000000 P 325bd09ed3ba4aeaad5e2ebc7947f96c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 23:18:26.693035 14367 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 325bd09ed3ba4aeaad5e2ebc7947f96c [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:18:26.694061 14367 raft_consensus.cc:504] T 00000000000000000000000000000000 P 325bd09ed3ba4aeaad5e2ebc7947f96c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "325bd09ed3ba4aeaad5e2ebc7947f96c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45695 } }
I20260923 23:18:26.694162 14367 leader_election.cc:302] T 00000000000000000000000000000000 P 325bd09ed3ba4aeaad5e2ebc7947f96c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 325bd09ed3ba4aeaad5e2ebc7947f96c; no voters:
I20260923 23:18:26.694322 14367 leader_election.cc:288] T 00000000000000000000000000000000 P 325bd09ed3ba4aeaad5e2ebc7947f96c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 23:18:26.694383 14372 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 325bd09ed3ba4aeaad5e2ebc7947f96c [term 1 FOLLOWER]: Leader election won for term 1
I20260923 23:18:26.694509 14372 raft_consensus.cc:686] T 00000000000000000000000000000000 P 325bd09ed3ba4aeaad5e2ebc7947f96c [term 1 LEADER]: Becoming Leader. State: Replica: 325bd09ed3ba4aeaad5e2ebc7947f96c, State: Running, Role: LEADER
I20260923 23:18:26.694517 14367 sys_catalog.cc:564] T 00000000000000000000000000000000 P 325bd09ed3ba4aeaad5e2ebc7947f96c [sys.catalog]: configured and running, proceeding with master startup.
I20260923 23:18:26.694648 14372 consensus_queue.cc:237] T 00000000000000000000000000000000 P 325bd09ed3ba4aeaad5e2ebc7947f96c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "325bd09ed3ba4aeaad5e2ebc7947f96c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45695 } }
I20260923 23:18:26.695731 14373 sys_catalog.cc:455] T 00000000000000000000000000000000 P 325bd09ed3ba4aeaad5e2ebc7947f96c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 325bd09ed3ba4aeaad5e2ebc7947f96c. Latest consensus state: current_term: 1 leader_uuid: "325bd09ed3ba4aeaad5e2ebc7947f96c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "325bd09ed3ba4aeaad5e2ebc7947f96c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45695 } } }
I20260923 23:18:26.695822 14373 sys_catalog.cc:458] T 00000000000000000000000000000000 P 325bd09ed3ba4aeaad5e2ebc7947f96c [sys.catalog]: This master's current role is: LEADER
W20260923 23:18:26.697052 14387 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 325bd09ed3ba4aeaad5e2ebc7947f96c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 23:18:26.697093 14387 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 23:18:26.697263 14388 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 23:18:26.697436 14388 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 23:18:26.698117 14374 sys_catalog.cc:455] T 00000000000000000000000000000000 P 325bd09ed3ba4aeaad5e2ebc7947f96c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "325bd09ed3ba4aeaad5e2ebc7947f96c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "325bd09ed3ba4aeaad5e2ebc7947f96c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45695 } } }
I20260923 23:18:26.698215 14374 sys_catalog.cc:458] T 00000000000000000000000000000000 P 325bd09ed3ba4aeaad5e2ebc7947f96c [sys.catalog]: This master's current role is: LEADER
I20260923 23:18:26.698643 14371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:26.692623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.172:56451 (local address 127.0.0.1:45695)
0923 23:18:26.692843 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:26.692851 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:26.692882 (+ 31us) server_negotiation.cc:407] Connection header received
0923 23:18:26.692986 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:26.692993 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:26.693048 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:26.693186 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:26.693331 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:26.694734 (+ 1403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:26.695792 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:26.696354 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:26.696505 (+ 151us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:26.697611 (+ 1106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:18:26.697615 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:18:26.697615 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 23:18:26.697822 (+ 207us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:18:26.697856 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 23:18:26.697870 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:18:26.697957 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:26.698410 (+ 453us) server_negotiation.cc:299] Negotiation successful
0923 23:18:26.698430 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":86,"threads_started":1}
I20260923 23:18:26.699282 14388 catalog_manager.cc:1347] Generated new cluster ID: 61372b0c92754f308c64d7dbc60ac45a
I20260923 23:18:26.699361 14388 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 23:18:26.699486 25487 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 23:18:26.704310 14371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:26.700934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.172:54907 (local address 127.0.0.1:45695)
0923 23:18:26.700973 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:26.700975 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:26.701105 (+ 130us) server_negotiation.cc:407] Connection header received
0923 23:18:26.701160 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:26.701161 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:26.701169 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:26.701237 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:26.701386 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:26.702066 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:26.703071 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:26.703374 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:26.703425 (+ 51us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:26.704085 (+ 660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:18:26.704086 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:18:26.704087 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 23:18:26.704090 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:18:26.704102 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 23:18:26.704106 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:18:26.704164 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:26.704237 (+ 73us) server_negotiation.cc:299] Negotiation successful
0923 23:18:26.704247 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260923 23:18:26.708279 14388 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 23:18:26.708711 14388 catalog_manager.cc:1504] Loading token signing keys...
I20260923 23:18:26.711547 14388 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 325bd09ed3ba4aeaad5e2ebc7947f96c: Generated new TSK 0
I20260923 23:18:26.711719 14388 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 23:18:26.719961 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 14299
W20260923 23:18:26.723901 14284 connection.cc:535] client connection to 127.0.0.1:45695 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45695): Connection reset by peer (error 104)
2026-09-23T23:18:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1231 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790205259943518-25487-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 23:18:26 dist-test-slave-754n krb5kdc[14398](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 23:18:26 dist-test-slave-754n krb5kdc[14398](info): set up 2 sockets
Sep 23 23:18:26 dist-test-slave-754n krb5kdc[14398](info): commencing operation
krb5kdc: starting...
W20260923 23:18:27.704039 25487 mini_kdc.cc:121] Time spent starting KDC: real 0.966s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790205259943518-25487-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790205259943518-25487-0/krb5kdc/test-user.keytab.
Sep 23 23:18:27 dist-test-slave-754n krb5kdc[14398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205507, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T23:18:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T23:18:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790205259943518-25487-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790205259943518-25487-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 23:18:27.807785 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790205259943518-25487-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790205259943518-25487-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790205259943518-25487-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34317
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790205259943518-25487-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37131
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34317
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:18:27.883615 14411 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:18:27.883797 14411 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:18:27.883817 14411 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:18:27.884846 14411 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 23:18:27.884884 14411 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:18:27.884897 14411 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:18:27.884938 14411 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 23:18:27.884963 14411 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 23:18:27.886225 14411 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37131
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790205259943518-25487-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790205259943518-25487-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34317
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790205259943518-25487-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34317
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14411
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790205259943518-25487-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:18:27.886377 14411 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:18:27.886584 14411 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:18:27.890785 14417 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:18:27.890792 14419 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:18:27.890797 14416 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:18:27.890974 14411 server_base.cc:1034] running on GCE node
I20260923 23:18:27.891189 14411 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:18:27.891470 14411 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:18:27.892627 14411 hybrid_clock.cc:648] HybridClock initialized: now 1790205507892613 us; error 26 us; skew 500 ppm
Sep 23 23:18:27 dist-test-slave-754n krb5kdc[14398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205507, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:18:27.894714 14411 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 23:18:27.895048 14411 webserver.cc:460] Webserver started at http://127.0.0.1:34261/ using document root <none> and password file <none>
I20260923 23:18:27.895251 14411 fs_manager.cc:359] Metadata directory not provided
I20260923 23:18:27.895308 14411 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:18:27.895434 14411 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:18:27.896399 14411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790205259943518-25487-0/minicluster-data/master-0/data/instance:
uuid: "dd470cecc9454e7fa8de33ef56f345ea"
format_stamp: "Formatted at 2026-09-23 23:18:27 on dist-test-slave-754n"
server_key: "a181e39a08728216437dfc11be585c35"
server_key_iv: "d77fb40eac70a2046556fe297cf8324e"
server_key_version: "encryptionkey@0"
I20260923 23:18:27.896708 14411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790205259943518-25487-0/minicluster-data/master-0/wal/instance:
uuid: "dd470cecc9454e7fa8de33ef56f345ea"
format_stamp: "Formatted at 2026-09-23 23:18:27 on dist-test-slave-754n"
server_key: "a181e39a08728216437dfc11be585c35"
server_key_iv: "d77fb40eac70a2046556fe297cf8324e"
server_key_version: "encryptionkey@0"
I20260923 23:18:27.897864 14411 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 23:18:27.898684 14426 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:18:27.898901 14411 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 23:18:27.898993 14411 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790205259943518-25487-0/minicluster-data/master-0/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790205259943518-25487-0/minicluster-data/master-0/wal
uuid: "dd470cecc9454e7fa8de33ef56f345ea"
format_stamp: "Formatted at 2026-09-23 23:18:27 on dist-test-slave-754n"
server_key: "a181e39a08728216437dfc11be585c35"
server_key_iv: "d77fb40eac70a2046556fe297cf8324e"
server_key_version: "encryptionkey@0"
I20260923 23:18:27.899071 14411 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790205259943518-25487-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790205259943518-25487-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790205259943518-25487-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:18:27.902179 14411 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:18:27.903793 14411 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:18:27.903945 14411 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:18:27.907233 14411 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34317
I20260923 23:18:27.907287 14478 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34317 every 8 connection(s)
I20260923 23:18:27.907577 14411 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
I20260923 23:18:27.908442 14479 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:18:27.910390 14479 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd470cecc9454e7fa8de33ef56f345ea: Bootstrap starting.
I20260923 23:18:27.910931 14479 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dd470cecc9454e7fa8de33ef56f345ea: Neither blocks nor log segments found. Creating new log.
I20260923 23:18:27.911156 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 14411
I20260923 23:18:27.911183 14479 log.cc:824] T 00000000000000000000000000000000 P dd470cecc9454e7fa8de33ef56f345ea: Log is configured to *not* fsync() on all Append() calls
I20260923 23:18:27.911273 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790205259943518-25487-0/minicluster-data/master-0/wal/instance
I20260923 23:18:27.911456 25487 external_mini_cluster.cc:1428] Setting key 8babc9b02258a83c6957d63b9472761f
I20260923 23:18:27.911970 14479 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd470cecc9454e7fa8de33ef56f345ea: No bootstrap required, opened a new log
I20260923 23:18:27.914070 14479 raft_consensus.cc:348] T 00000000000000000000000000000000 P dd470cecc9454e7fa8de33ef56f345ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd470cecc9454e7fa8de33ef56f345ea" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34317 } }
I20260923 23:18:27.914232 14479 raft_consensus.cc:374] T 00000000000000000000000000000000 P dd470cecc9454e7fa8de33ef56f345ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:18:27.914302 14479 raft_consensus.cc:729] T 00000000000000000000000000000000 P dd470cecc9454e7fa8de33ef56f345ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd470cecc9454e7fa8de33ef56f345ea, State: Initialized, Role: FOLLOWER
I20260923 23:18:27.914412 14479 consensus_queue.cc:260] T 00000000000000000000000000000000 P dd470cecc9454e7fa8de33ef56f345ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd470cecc9454e7fa8de33ef56f345ea" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34317 } }
I20260923 23:18:27.914513 14479 raft_consensus.cc:388] T 00000000000000000000000000000000 P dd470cecc9454e7fa8de33ef56f345ea [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 23:18:27.914541 14483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:27.911919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.172:56499 (local address 127.0.0.1:34317)
0923 23:18:27.912185 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:27.912193 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:27.912224 (+ 31us) server_negotiation.cc:407] Connection header received
0923 23:18:27.913173 (+ 949us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:27.913182 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:27.913214 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:27.913355 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:27.913909 (+ 554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:18:27.913912 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:18:27.913913 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 23:18:27.913917 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:18:27.913965 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 23:18:27.913980 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:18:27.914053 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:27.914128 (+ 75us) server_negotiation.cc:299] Negotiation successful
0923 23:18:27.914167 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":74,"threads_started":1}
I20260923 23:18:27.914548 14479 raft_consensus.cc:482] T 00000000000000000000000000000000 P dd470cecc9454e7fa8de33ef56f345ea [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 23:18:27.914613 14479 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dd470cecc9454e7fa8de33ef56f345ea [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:18:27.915341 14479 raft_consensus.cc:504] T 00000000000000000000000000000000 P dd470cecc9454e7fa8de33ef56f345ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd470cecc9454e7fa8de33ef56f345ea" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34317 } }
I20260923 23:18:27.915469 25487 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 23:18:27.915462 14479 leader_election.cc:302] T 00000000000000000000000000000000 P dd470cecc9454e7fa8de33ef56f345ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dd470cecc9454e7fa8de33ef56f345ea; no voters:
I20260923 23:18:27.915606 14479 leader_election.cc:288] T 00000000000000000000000000000000 P dd470cecc9454e7fa8de33ef56f345ea [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 23:18:27.915633 14484 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dd470cecc9454e7fa8de33ef56f345ea [term 1 FOLLOWER]: Leader election won for term 1
I20260923 23:18:27.915740 14484 raft_consensus.cc:686] T 00000000000000000000000000000000 P dd470cecc9454e7fa8de33ef56f345ea [term 1 LEADER]: Becoming Leader. State: Replica: dd470cecc9454e7fa8de33ef56f345ea, State: Running, Role: LEADER
I20260923 23:18:27.915840 14479 sys_catalog.cc:564] T 00000000000000000000000000000000 P dd470cecc9454e7fa8de33ef56f345ea [sys.catalog]: configured and running, proceeding with master startup.
I20260923 23:18:27.915895 14484 consensus_queue.cc:237] T 00000000000000000000000000000000 P dd470cecc9454e7fa8de33ef56f345ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd470cecc9454e7fa8de33ef56f345ea" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34317 } }
I20260923 23:18:27.916229 14484 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd470cecc9454e7fa8de33ef56f345ea [sys.catalog]: SysCatalogTable state changed. Reason: New leader dd470cecc9454e7fa8de33ef56f345ea. Latest consensus state: current_term: 1 leader_uuid: "dd470cecc9454e7fa8de33ef56f345ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd470cecc9454e7fa8de33ef56f345ea" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34317 } } }
I20260923 23:18:27.916471 14484 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd470cecc9454e7fa8de33ef56f345ea [sys.catalog]: This master's current role is: LEADER
I20260923 23:18:27.916385 14485 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd470cecc9454e7fa8de33ef56f345ea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dd470cecc9454e7fa8de33ef56f345ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd470cecc9454e7fa8de33ef56f345ea" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34317 } } }
I20260923 23:18:27.916805 14485 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd470cecc9454e7fa8de33ef56f345ea [sys.catalog]: This master's current role is: LEADER
I20260923 23:18:27.916906 14496 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 23:18:27.917383 14496 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 23:18:27.918759 14496 catalog_manager.cc:1347] Generated new cluster ID: 138ae1440d734a608c3e9ac28672e15c
I20260923 23:18:27.918803 14496 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 23:18:27.924325 14483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:27.918090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.172:46231 (local address 127.0.0.1:34317)
0923 23:18:27.918116 (+ 26us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:27.918117 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:27.918328 (+ 211us) server_negotiation.cc:407] Connection header received
0923 23:18:27.918414 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:27.918415 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:27.918419 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:27.918472 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:27.924141 (+ 5669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:18:27.924142 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:18:27.924142 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 23:18:27.924146 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:18:27.924157 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 23:18:27.924161 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:18:27.924215 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:27.924266 (+ 51us) server_negotiation.cc:299] Negotiation successful
0923 23:18:27.924276 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13}
I20260923 23:18:27.929093 14496 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 23:18:27.929260 14496 catalog_manager.cc:1504] Loading token signing keys...
I20260923 23:18:27.944263 14496 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dd470cecc9454e7fa8de33ef56f345ea: Generated new TSK 0
I20260923 23:18:27.944422 14496 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 23:18:27.946316 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 14411
2026-09-23T23:18:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1226 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790205259943518-25487-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 23:18:27 dist-test-slave-754n krb5kdc[14510](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 23:18:27 dist-test-slave-754n krb5kdc[14510](info): set up 2 sockets
Sep 23 23:18:27 dist-test-slave-754n krb5kdc[14510](info): commencing operation
krb5kdc: starting...
W20260923 23:18:28.959734 25487 mini_kdc.cc:121] Time spent starting KDC: real 0.996s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790205259943518-25487-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790205259943518-25487-0/krb5kdc/test-user.keytab.
Sep 23 23:18:29 dist-test-slave-754n krb5kdc[14510](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205509, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T23:18:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T23:18:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.227.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.227.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.227.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab.
Entry for principal kudu/127.24.227.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab.
I20260923 23:18:29.067346 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790205259943518-25487-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790205259943518-25487-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790205259943518-25487-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.254:40889
--webserver_interface=127.24.227.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab
--principal=kudu/127.24.227.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:32981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.227.254:40889
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:18:29.143165 14523 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:18:29.143362 14523 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:18:29.143383 14523 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:18:29.144449 14523 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 23:18:29.144488 14523 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:18:29.144502 14523 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:18:29.144516 14523 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 23:18:29.144544 14523 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 23:18:29.145867 14523 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:32981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790205259943518-25487-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790205259943518-25487-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.227.254:40889
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab
--principal=kudu/127.24.227.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.254:40889
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.227.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14523
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790205259943518-25487-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:18:29.146039 14523 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:18:29.146173 14523 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:18:29.150481 14531 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:18:29.150470 14529 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:18:29.150460 14528 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:18:29.150570 14523 server_base.cc:1034] running on GCE node
I20260923 23:18:29.150902 14523 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:18:29.151170 14523 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:18:29.152302 14523 hybrid_clock.cc:648] HybridClock initialized: now 1790205509152290 us; error 28 us; skew 500 ppm
Sep 23 23:18:29 dist-test-slave-754n krb5kdc[14510](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205509, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:18:29.154531 14523 init.cc:377] Logged in from keytab as kudu/127.24.227.254@KRBTEST.COM (short username kudu)
I20260923 23:18:29.154938 14523 webserver.cc:460] Webserver started at http://127.24.227.254:45331/ using document root <none> and password file <none>
I20260923 23:18:29.155144 14523 fs_manager.cc:359] Metadata directory not provided
I20260923 23:18:29.155185 14523 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:18:29.155331 14523 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:18:29.156298 14523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790205259943518-25487-0/minicluster-data/master-0/data/instance:
uuid: "eb84dface78c4a7cb201fe32b885c654"
format_stamp: "Formatted at 2026-09-23 23:18:29 on dist-test-slave-754n"
server_key: "c44c66c635726e9dfe05a2ef779361cd"
server_key_iv: "c217a7c411115a38b79133a4404a7141"
server_key_version: "encryptionkey@0"
I20260923 23:18:29.156647 14523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790205259943518-25487-0/minicluster-data/master-0/wal/instance:
uuid: "eb84dface78c4a7cb201fe32b885c654"
format_stamp: "Formatted at 2026-09-23 23:18:29 on dist-test-slave-754n"
server_key: "c44c66c635726e9dfe05a2ef779361cd"
server_key_iv: "c217a7c411115a38b79133a4404a7141"
server_key_version: "encryptionkey@0"
I20260923 23:18:29.157866 14523 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 23:18:29.158552 14538 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:18:29.158712 14523 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 23:18:29.158804 14523 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790205259943518-25487-0/minicluster-data/master-0/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790205259943518-25487-0/minicluster-data/master-0/wal
uuid: "eb84dface78c4a7cb201fe32b885c654"
format_stamp: "Formatted at 2026-09-23 23:18:29 on dist-test-slave-754n"
server_key: "c44c66c635726e9dfe05a2ef779361cd"
server_key_iv: "c217a7c411115a38b79133a4404a7141"
server_key_version: "encryptionkey@0"
I20260923 23:18:29.158874 14523 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790205259943518-25487-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790205259943518-25487-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790205259943518-25487-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:18:29.168804 14523 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:18:29.170469 14523 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:18:29.170614 14523 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:18:29.174396 14523 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.254:40889
I20260923 23:18:29.174423 14590 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.254:40889 every 8 connection(s)
I20260923 23:18:29.174738 14523 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
I20260923 23:18:29.175551 14591 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:18:29.177624 14591 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb84dface78c4a7cb201fe32b885c654: Bootstrap starting.
I20260923 23:18:29.178148 14591 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eb84dface78c4a7cb201fe32b885c654: Neither blocks nor log segments found. Creating new log.
I20260923 23:18:29.178413 14591 log.cc:824] T 00000000000000000000000000000000 P eb84dface78c4a7cb201fe32b885c654: Log is configured to *not* fsync() on all Append() calls
I20260923 23:18:29.179039 14591 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb84dface78c4a7cb201fe32b885c654: No bootstrap required, opened a new log
I20260923 23:18:29.180626 14591 raft_consensus.cc:348] T 00000000000000000000000000000000 P eb84dface78c4a7cb201fe32b885c654 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb84dface78c4a7cb201fe32b885c654" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 40889 } }
I20260923 23:18:29.180768 14591 raft_consensus.cc:374] T 00000000000000000000000000000000 P eb84dface78c4a7cb201fe32b885c654 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:18:29.180809 14591 raft_consensus.cc:729] T 00000000000000000000000000000000 P eb84dface78c4a7cb201fe32b885c654 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb84dface78c4a7cb201fe32b885c654, State: Initialized, Role: FOLLOWER
I20260923 23:18:29.180972 14591 consensus_queue.cc:260] T 00000000000000000000000000000000 P eb84dface78c4a7cb201fe32b885c654 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb84dface78c4a7cb201fe32b885c654" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 40889 } }
I20260923 23:18:29.181052 14591 raft_consensus.cc:388] T 00000000000000000000000000000000 P eb84dface78c4a7cb201fe32b885c654 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 23:18:29.181102 14591 raft_consensus.cc:482] T 00000000000000000000000000000000 P eb84dface78c4a7cb201fe32b885c654 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 23:18:29.181166 14591 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eb84dface78c4a7cb201fe32b885c654 [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:18:29.181334 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 14523
I20260923 23:18:29.181452 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790205259943518-25487-0/minicluster-data/master-0/wal/instance
I20260923 23:18:29.181629 25487 external_mini_cluster.cc:1428] Setting key ee664cec1f5844b7d42f88c55db94be7
I20260923 23:18:29.182221 14591 raft_consensus.cc:504] T 00000000000000000000000000000000 P eb84dface78c4a7cb201fe32b885c654 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb84dface78c4a7cb201fe32b885c654" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 40889 } }
I20260923 23:18:29.182323 14591 leader_election.cc:302] T 00000000000000000000000000000000 P eb84dface78c4a7cb201fe32b885c654 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eb84dface78c4a7cb201fe32b885c654; no voters:
I20260923 23:18:29.182469 14591 leader_election.cc:288] T 00000000000000000000000000000000 P eb84dface78c4a7cb201fe32b885c654 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 23:18:29.182503 14596 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eb84dface78c4a7cb201fe32b885c654 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 23:18:29.182641 14596 raft_consensus.cc:686] T 00000000000000000000000000000000 P eb84dface78c4a7cb201fe32b885c654 [term 1 LEADER]: Becoming Leader. State: Replica: eb84dface78c4a7cb201fe32b885c654, State: Running, Role: LEADER
I20260923 23:18:29.182689 14591 sys_catalog.cc:564] T 00000000000000000000000000000000 P eb84dface78c4a7cb201fe32b885c654 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 23:18:29.182755 14596 consensus_queue.cc:237] T 00000000000000000000000000000000 P eb84dface78c4a7cb201fe32b885c654 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb84dface78c4a7cb201fe32b885c654" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 40889 } }
I20260923 23:18:29.183105 14596 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb84dface78c4a7cb201fe32b885c654 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eb84dface78c4a7cb201fe32b885c654. Latest consensus state: current_term: 1 leader_uuid: "eb84dface78c4a7cb201fe32b885c654" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb84dface78c4a7cb201fe32b885c654" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 40889 } } }
I20260923 23:18:29.183115 14597 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb84dface78c4a7cb201fe32b885c654 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eb84dface78c4a7cb201fe32b885c654" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb84dface78c4a7cb201fe32b885c654" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 40889 } } }
I20260923 23:18:29.183251 14596 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb84dface78c4a7cb201fe32b885c654 [sys.catalog]: This master's current role is: LEADER
I20260923 23:18:29.183260 14597 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb84dface78c4a7cb201fe32b885c654 [sys.catalog]: This master's current role is: LEADER
I20260923 23:18:29.183516 14605 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 23:18:29.184566 14605 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 23:18:29.186213 14605 catalog_manager.cc:1347] Generated new cluster ID: 90d2172bc809450d90ab30efab2cbf0b
I20260923 23:18:29.186264 14605 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 23:18:29 dist-test-slave-754n krb5kdc[14510](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205509, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
I20260923 23:18:29.192284 14595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:29.182056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49263 (local address 127.24.227.254:40889)
0923 23:18:29.182265 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:29.182274 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:29.182308 (+ 34us) server_negotiation.cc:407] Connection header received
0923 23:18:29.182418 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:29.182425 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:29.182492 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:29.182630 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:29.183558 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:29.185186 (+ 1628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:29.186188 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:29.186663 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:29.186812 (+ 149us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:29.189170 (+ 2358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:18:29.189173 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:18:29.189174 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:18:29.189185 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:18:29.190786 (+ 1601us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:29.191175 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:29.191180 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:29.191180 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:29.191271 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:29.191536 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:29.191537 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:29.191537 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:29.191745 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:18:29.191950 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:29.192030 (+ 80us) server_negotiation.cc:299] Negotiation successful
0923 23:18:29.192126 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":103,"threads_started":1}
I20260923 23:18:29.193113 25487 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 23:18:29.200201 14595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:29.194467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47371 (local address 127.24.227.254:40889)
0923 23:18:29.194502 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:29.194504 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:29.194512 (+ 8us) server_negotiation.cc:407] Connection header received
0923 23:18:29.194569 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:29.194570 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:29.194577 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:29.194630 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:29.195118 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:29.195605 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:29.196363 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:29.196599 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:29.196689 (+ 90us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:29.197754 (+ 1065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:18:29.197755 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:18:29.197756 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:18:29.197763 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:18:29.199052 (+ 1289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:29.199403 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:29.199404 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:29.199404 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:29.199454 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:29.199674 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:29.199676 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:29.199676 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:29.199793 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:18:29.199935 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:29.200057 (+ 122us) server_negotiation.cc:299] Negotiation successful
0923 23:18:29.200115 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260923 23:18:29.202299 14605 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 23:18:29.202950 14605 catalog_manager.cc:1504] Loading token signing keys...
I20260923 23:18:29.208361 14605 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eb84dface78c4a7cb201fe32b885c654: Generated new TSK 0
I20260923 23:18:29.208509 14605 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 23:18:29.210783 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 14523
W20260923 23:18:29.214397 14508 connection.cc:535] client connection to 127.24.227.254:40889 recv error: Network error: failed to read from TLS socket (remote: 127.24.227.254:40889): Connection reset by peer (error 104)
2026-09-23T23:18:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1264 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790205259943518-25487-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 23:18:29 dist-test-slave-754n krb5kdc[14622](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 23:18:29 dist-test-slave-754n krb5kdc[14622](info): set up 2 sockets
Sep 23 23:18:29 dist-test-slave-754n krb5kdc[14622](info): commencing operation
krb5kdc: starting...
W20260923 23:18:30.207741 25487 mini_kdc.cc:121] Time spent starting KDC: real 0.980s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790205259943518-25487-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790205259943518-25487-0/krb5kdc/test-user.keytab.
Sep 23 23:18:30 dist-test-slave-754n krb5kdc[14622](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205510, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T23:18:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T23:18:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.227.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.227.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.227.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab.
Entry for principal kudu/127.24.227.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab.
I20260923 23:18:30.314680 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790205259943518-25487-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790205259943518-25487-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790205259943518-25487-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.254:37647
--webserver_interface=127.24.227.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab
--principal=kudu/127.24.227.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:42345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.227.254:37647
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:18:30.389806 14635 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:18:30.390012 14635 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:18:30.390046 14635 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:18:30.391137 14635 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 23:18:30.391203 14635 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:18:30.391228 14635 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:18:30.391256 14635 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 23:18:30.391279 14635 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 23:18:30.392711 14635 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:42345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790205259943518-25487-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790205259943518-25487-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.227.254:37647
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab
--principal=kudu/127.24.227.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.254:37647
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.227.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14635
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790205259943518-25487-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:18:30.392961 14635 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:18:30.393177 14635 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:18:30.397518 14643 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:18:30.397562 14641 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:18:30.397562 14640 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:18:30.397738 14635 server_base.cc:1034] running on GCE node
I20260923 23:18:30.398066 14635 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:18:30.398329 14635 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:18:30.399505 14635 hybrid_clock.cc:648] HybridClock initialized: now 1790205510399504 us; error 39 us; skew 500 ppm
Sep 23 23:18:30 dist-test-slave-754n krb5kdc[14622](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205510, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:18:30.401827 14635 init.cc:377] Logged in from keytab as kudu/127.24.227.254@KRBTEST.COM (short username kudu)
I20260923 23:18:30.402268 14635 webserver.cc:460] Webserver started at http://127.24.227.254:33775/ using document root <none> and password file <none>
I20260923 23:18:30.402478 14635 fs_manager.cc:359] Metadata directory not provided
I20260923 23:18:30.402542 14635 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:18:30.402686 14635 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:18:30.403684 14635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790205259943518-25487-0/minicluster-data/master-0/data/instance:
uuid: "8fce68263d3142d0909f1804f5d895ac"
format_stamp: "Formatted at 2026-09-23 23:18:30 on dist-test-slave-754n"
server_key: "5fe62f0c7b67c95864166122942d1b60"
server_key_iv: "140535042f65d9c1ea8ef6696e14d622"
server_key_version: "encryptionkey@0"
I20260923 23:18:30.404042 14635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790205259943518-25487-0/minicluster-data/master-0/wal/instance:
uuid: "8fce68263d3142d0909f1804f5d895ac"
format_stamp: "Formatted at 2026-09-23 23:18:30 on dist-test-slave-754n"
server_key: "5fe62f0c7b67c95864166122942d1b60"
server_key_iv: "140535042f65d9c1ea8ef6696e14d622"
server_key_version: "encryptionkey@0"
I20260923 23:18:30.405273 14635 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 23:18:30.405911 14650 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:18:30.406065 14635 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 23:18:30.406157 14635 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790205259943518-25487-0/minicluster-data/master-0/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790205259943518-25487-0/minicluster-data/master-0/wal
uuid: "8fce68263d3142d0909f1804f5d895ac"
format_stamp: "Formatted at 2026-09-23 23:18:30 on dist-test-slave-754n"
server_key: "5fe62f0c7b67c95864166122942d1b60"
server_key_iv: "140535042f65d9c1ea8ef6696e14d622"
server_key_version: "encryptionkey@0"
I20260923 23:18:30.406234 14635 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790205259943518-25487-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790205259943518-25487-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790205259943518-25487-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:18:30.435123 14635 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:18:30.437291 14635 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:18:30.437448 14635 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:18:30.441258 14635 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.254:37647
I20260923 23:18:30.441289 14702 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.254:37647 every 8 connection(s)
I20260923 23:18:30.441604 14635 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
I20260923 23:18:30.442409 14703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:18:30.444525 14703 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8fce68263d3142d0909f1804f5d895ac: Bootstrap starting.
I20260923 23:18:30.445068 14703 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8fce68263d3142d0909f1804f5d895ac: Neither blocks nor log segments found. Creating new log.
I20260923 23:18:30.445331 14703 log.cc:824] T 00000000000000000000000000000000 P 8fce68263d3142d0909f1804f5d895ac: Log is configured to *not* fsync() on all Append() calls
I20260923 23:18:30.445922 14703 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8fce68263d3142d0909f1804f5d895ac: No bootstrap required, opened a new log
I20260923 23:18:30.447326 14703 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8fce68263d3142d0909f1804f5d895ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fce68263d3142d0909f1804f5d895ac" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 37647 } }
I20260923 23:18:30.447445 14703 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8fce68263d3142d0909f1804f5d895ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:18:30.447495 14703 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8fce68263d3142d0909f1804f5d895ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fce68263d3142d0909f1804f5d895ac, State: Initialized, Role: FOLLOWER
I20260923 23:18:30.447660 14703 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8fce68263d3142d0909f1804f5d895ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fce68263d3142d0909f1804f5d895ac" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 37647 } }
I20260923 23:18:30.447767 14703 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8fce68263d3142d0909f1804f5d895ac [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 23:18:30.447804 14703 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8fce68263d3142d0909f1804f5d895ac [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 23:18:30.447865 14703 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8fce68263d3142d0909f1804f5d895ac [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:18:30.448331 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 14635
I20260923 23:18:30.448475 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790205259943518-25487-0/minicluster-data/master-0/wal/instance
I20260923 23:18:30.448650 25487 external_mini_cluster.cc:1428] Setting key 75cc0526514de3724e3c4b08be07314a
I20260923 23:18:30.448827 14703 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8fce68263d3142d0909f1804f5d895ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fce68263d3142d0909f1804f5d895ac" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 37647 } }
I20260923 23:18:30.448915 14703 leader_election.cc:302] T 00000000000000000000000000000000 P 8fce68263d3142d0909f1804f5d895ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8fce68263d3142d0909f1804f5d895ac; no voters:
I20260923 23:18:30.449203 14703 leader_election.cc:288] T 00000000000000000000000000000000 P 8fce68263d3142d0909f1804f5d895ac [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 23:18:30.449235 14707 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8fce68263d3142d0909f1804f5d895ac [term 1 FOLLOWER]: Leader election won for term 1
I20260923 23:18:30.449436 14707 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8fce68263d3142d0909f1804f5d895ac [term 1 LEADER]: Becoming Leader. State: Replica: 8fce68263d3142d0909f1804f5d895ac, State: Running, Role: LEADER
I20260923 23:18:30.449529 14703 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8fce68263d3142d0909f1804f5d895ac [sys.catalog]: configured and running, proceeding with master startup.
I20260923 23:18:30.449579 14707 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8fce68263d3142d0909f1804f5d895ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fce68263d3142d0909f1804f5d895ac" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 37647 } }
I20260923 23:18:30.450002 14707 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fce68263d3142d0909f1804f5d895ac [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8fce68263d3142d0909f1804f5d895ac. Latest consensus state: current_term: 1 leader_uuid: "8fce68263d3142d0909f1804f5d895ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fce68263d3142d0909f1804f5d895ac" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 37647 } } }
I20260923 23:18:30.450012 14709 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fce68263d3142d0909f1804f5d895ac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8fce68263d3142d0909f1804f5d895ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fce68263d3142d0909f1804f5d895ac" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 37647 } } }
I20260923 23:18:30.450157 14709 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fce68263d3142d0909f1804f5d895ac [sys.catalog]: This master's current role is: LEADER
I20260923 23:18:30.450335 14707 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fce68263d3142d0909f1804f5d895ac [sys.catalog]: This master's current role is: LEADER
W20260923 23:18:30.451424 14724 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8fce68263d3142d0909f1804f5d895ac: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 23:18:30.451520 14724 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 23:18:30.451643 14717 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 23:18:30.451839 14717 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 23:18:30.453382 14717 catalog_manager.cc:1347] Generated new cluster ID: c5e9e258390b4f73ac169f29aca9c4e7
I20260923 23:18:30.453430 14717 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 23:18:30.454458 14708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:30.449075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42657 (local address 127.24.227.254:37647)
0923 23:18:30.449347 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:30.449356 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:30.449399 (+ 43us) server_negotiation.cc:407] Connection header received
0923 23:18:30.449515 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:30.449522 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:30.449576 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:30.449956 (+ 380us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:30.450044 (+ 88us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:30.451434 (+ 1390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:30.452540 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:30.453081 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:30.453184 (+ 103us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:30.453819 (+ 635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:18:30.453822 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:18:30.453823 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 23:18:30.454016 (+ 193us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:18:30.454050 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 23:18:30.454063 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:18:30.454140 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:30.454255 (+ 115us) server_negotiation.cc:299] Negotiation successful
0923 23:18:30.454277 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":215,"thread_start_us":149,"threads_started":1}
I20260923 23:18:30.455372 25487 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 23:18:30.460201 14708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:30.456896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51797 (local address 127.24.227.254:37647)
0923 23:18:30.456929 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:30.456931 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:30.456996 (+ 65us) server_negotiation.cc:407] Connection header received
0923 23:18:30.457065 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:30.457065 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:30.457083 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:30.457130 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:30.457265 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:30.457941 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:30.458951 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:30.459256 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:30.459309 (+ 53us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:30.459937 (+ 628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:18:30.459938 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:18:30.459939 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 23:18:30.459943 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:18:30.459960 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 23:18:30.459964 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:18:30.460017 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:30.460125 (+ 108us) server_negotiation.cc:299] Negotiation successful
0923 23:18:30.460136 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260923 23:18:30.462386 14717 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 23:18:30.463065 14717 catalog_manager.cc:1504] Loading token signing keys...
I20260923 23:18:30.470798 14717 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8fce68263d3142d0909f1804f5d895ac: Generated new TSK 0
I20260923 23:18:30.470943 14717 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 23:18:30.474308 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 14635
W20260923 23:18:30.478379 14620 connection.cc:535] client connection to 127.24.227.254:37647 recv error: Network error: failed to read from TLS socket (remote: 127.24.227.254:37647): Connection reset by peer (error 104)
2026-09-23T23:18:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1263 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790205259943518-25487-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 23:18:30 dist-test-slave-754n krb5kdc[14734](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 23:18:30 dist-test-slave-754n krb5kdc[14734](info): set up 2 sockets
Sep 23 23:18:30 dist-test-slave-754n krb5kdc[14734](info): commencing operation
krb5kdc: starting...
W20260923 23:18:31.476006 25487 mini_kdc.cc:121] Time spent starting KDC: real 0.984s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790205259943518-25487-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790205259943518-25487-0/krb5kdc/test-user.keytab.
Sep 23 23:18:31 dist-test-slave-754n krb5kdc[14734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205511, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T23:18:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T23:18:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.227.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.227.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.227.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab.
Entry for principal kudu/127.24.227.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab.
I20260923 23:18:31.585570 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790205259943518-25487-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790205259943518-25487-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790205259943518-25487-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.254:45489
--webserver_interface=127.24.227.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab
--principal=kudu/127.24.227.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:36109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.227.254:45489
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:18:31.660910 14747 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:18:31.661093 14747 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:18:31.661115 14747 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:18:31.662214 14747 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 23:18:31.662254 14747 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:18:31.662268 14747 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:18:31.662320 14747 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 23:18:31.662354 14747 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 23:18:31.663769 14747 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:36109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790205259943518-25487-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790205259943518-25487-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.227.254:45489
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab
--principal=kudu/127.24.227.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.254:45489
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.227.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14747
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790205259943518-25487-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:18:31.664013 14747 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:18:31.664227 14747 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:18:31.668493 14752 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:18:31.668502 14755 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:18:31.668563 14753 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:18:31.668828 14747 server_base.cc:1034] running on GCE node
I20260923 23:18:31.669054 14747 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:18:31.669346 14747 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:18:31.670507 14747 hybrid_clock.cc:648] HybridClock initialized: now 1790205511670503 us; error 35 us; skew 500 ppm
Sep 23 23:18:31 dist-test-slave-754n krb5kdc[14734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205511, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:18:31.672878 14747 init.cc:377] Logged in from keytab as kudu/127.24.227.254@KRBTEST.COM (short username kudu)
I20260923 23:18:31.673286 14747 webserver.cc:460] Webserver started at http://127.24.227.254:34613/ using document root <none> and password file <none>
I20260923 23:18:31.673480 14747 fs_manager.cc:359] Metadata directory not provided
I20260923 23:18:31.673517 14747 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:18:31.673609 14747 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:18:31.674432 14747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790205259943518-25487-0/minicluster-data/master-0/data/instance:
uuid: "4a7505d9b24d4c6d8c34a0ed87bdbbfe"
format_stamp: "Formatted at 2026-09-23 23:18:31 on dist-test-slave-754n"
server_key: "6b578a7204ca243896e72f27d627f156"
server_key_iv: "d5de00c32235c21c938ec0340197e4b9"
server_key_version: "encryptionkey@0"
I20260923 23:18:31.674690 14747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790205259943518-25487-0/minicluster-data/master-0/wal/instance:
uuid: "4a7505d9b24d4c6d8c34a0ed87bdbbfe"
format_stamp: "Formatted at 2026-09-23 23:18:31 on dist-test-slave-754n"
server_key: "6b578a7204ca243896e72f27d627f156"
server_key_iv: "d5de00c32235c21c938ec0340197e4b9"
server_key_version: "encryptionkey@0"
I20260923 23:18:31.675923 14747 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 23:18:31.676560 14762 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:18:31.676738 14747 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 23:18:31.676817 14747 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790205259943518-25487-0/minicluster-data/master-0/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790205259943518-25487-0/minicluster-data/master-0/wal
uuid: "4a7505d9b24d4c6d8c34a0ed87bdbbfe"
format_stamp: "Formatted at 2026-09-23 23:18:31 on dist-test-slave-754n"
server_key: "6b578a7204ca243896e72f27d627f156"
server_key_iv: "d5de00c32235c21c938ec0340197e4b9"
server_key_version: "encryptionkey@0"
I20260923 23:18:31.676877 14747 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790205259943518-25487-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790205259943518-25487-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790205259943518-25487-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:18:31.680003 14747 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:18:31.681883 14747 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:18:31.681978 14747 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:18:31.685451 14747 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.254:45489
I20260923 23:18:31.685520 14814 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.254:45489 every 8 connection(s)
I20260923 23:18:31.685766 14747 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
I20260923 23:18:31.686671 14815 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:18:31.688627 14815 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a7505d9b24d4c6d8c34a0ed87bdbbfe: Bootstrap starting.
I20260923 23:18:31.689149 14815 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4a7505d9b24d4c6d8c34a0ed87bdbbfe: Neither blocks nor log segments found. Creating new log.
I20260923 23:18:31.689503 14815 log.cc:824] T 00000000000000000000000000000000 P 4a7505d9b24d4c6d8c34a0ed87bdbbfe: Log is configured to *not* fsync() on all Append() calls
I20260923 23:18:31.689546 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 14747
I20260923 23:18:31.689644 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790205259943518-25487-0/minicluster-data/master-0/wal/instance
I20260923 23:18:31.689818 25487 external_mini_cluster.cc:1428] Setting key 417da0582ee00e12bccd050dfc0ddb7c
I20260923 23:18:31.690553 14815 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a7505d9b24d4c6d8c34a0ed87bdbbfe: No bootstrap required, opened a new log
I20260923 23:18:31.692035 14819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:31.690243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38315 (local address 127.24.227.254:45489)
0923 23:18:31.690416 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:31.690425 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:31.690455 (+ 30us) server_negotiation.cc:407] Connection header received
0923 23:18:31.690900 (+ 445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:31.690906 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:31.690930 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:31.691059 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:31.691646 (+ 587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:18:31.691650 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:18:31.691650 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 23:18:31.691654 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:18:31.691693 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 23:18:31.691702 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:18:31.691792 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:31.691825 (+ 33us) server_negotiation.cc:299] Negotiation successful
0923 23:18:31.691886 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":78,"threads_started":1}
I20260923 23:18:31.693004 25487 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 23:18:31.692987 14815 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4a7505d9b24d4c6d8c34a0ed87bdbbfe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a7505d9b24d4c6d8c34a0ed87bdbbfe" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 45489 } }
I20260923 23:18:31.693146 14815 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4a7505d9b24d4c6d8c34a0ed87bdbbfe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:18:31.693189 14815 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4a7505d9b24d4c6d8c34a0ed87bdbbfe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a7505d9b24d4c6d8c34a0ed87bdbbfe, State: Initialized, Role: FOLLOWER
I20260923 23:18:31.693303 14815 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4a7505d9b24d4c6d8c34a0ed87bdbbfe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a7505d9b24d4c6d8c34a0ed87bdbbfe" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 45489 } }
I20260923 23:18:31.693387 14815 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4a7505d9b24d4c6d8c34a0ed87bdbbfe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 23:18:31.693430 14815 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4a7505d9b24d4c6d8c34a0ed87bdbbfe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 23:18:31.693477 14815 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4a7505d9b24d4c6d8c34a0ed87bdbbfe [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:18:31.694010 14815 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4a7505d9b24d4c6d8c34a0ed87bdbbfe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a7505d9b24d4c6d8c34a0ed87bdbbfe" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 45489 } }
I20260923 23:18:31.694109 14815 leader_election.cc:302] T 00000000000000000000000000000000 P 4a7505d9b24d4c6d8c34a0ed87bdbbfe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4a7505d9b24d4c6d8c34a0ed87bdbbfe; no voters:
I20260923 23:18:31.694268 14815 leader_election.cc:288] T 00000000000000000000000000000000 P 4a7505d9b24d4c6d8c34a0ed87bdbbfe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 23:18:31.694296 14824 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4a7505d9b24d4c6d8c34a0ed87bdbbfe [term 1 FOLLOWER]: Leader election won for term 1
I20260923 23:18:31.694471 14824 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4a7505d9b24d4c6d8c34a0ed87bdbbfe [term 1 LEADER]: Becoming Leader. State: Replica: 4a7505d9b24d4c6d8c34a0ed87bdbbfe, State: Running, Role: LEADER
I20260923 23:18:31.694507 14815 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4a7505d9b24d4c6d8c34a0ed87bdbbfe [sys.catalog]: configured and running, proceeding with master startup.
I20260923 23:18:31.694584 14824 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4a7505d9b24d4c6d8c34a0ed87bdbbfe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a7505d9b24d4c6d8c34a0ed87bdbbfe" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 45489 } }
I20260923 23:18:31.694902 14824 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a7505d9b24d4c6d8c34a0ed87bdbbfe [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4a7505d9b24d4c6d8c34a0ed87bdbbfe. Latest consensus state: current_term: 1 leader_uuid: "4a7505d9b24d4c6d8c34a0ed87bdbbfe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a7505d9b24d4c6d8c34a0ed87bdbbfe" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 45489 } } }
I20260923 23:18:31.694914 14825 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a7505d9b24d4c6d8c34a0ed87bdbbfe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4a7505d9b24d4c6d8c34a0ed87bdbbfe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a7505d9b24d4c6d8c34a0ed87bdbbfe" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 45489 } } }
I20260923 23:18:31.695042 14824 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a7505d9b24d4c6d8c34a0ed87bdbbfe [sys.catalog]: This master's current role is: LEADER
I20260923 23:18:31.695097 14825 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a7505d9b24d4c6d8c34a0ed87bdbbfe [sys.catalog]: This master's current role is: LEADER
I20260923 23:18:31.695467 14834 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 23:18:31.695937 14834 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 23:18:31.696966 14819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:31.694393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41147 (local address 127.24.227.254:45489)
0923 23:18:31.694426 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:31.694428 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:31.695919 (+ 1491us) server_negotiation.cc:407] Connection header received
0923 23:18:31.696031 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:31.696032 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:31.696036 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:31.696063 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:31.696630 (+ 567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:18:31.696632 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:18:31.696632 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 23:18:31.696636 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:18:31.696649 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 23:18:31.696653 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:18:31.696715 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:31.696901 (+ 186us) server_negotiation.cc:299] Negotiation successful
0923 23:18:31.696913 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260923 23:18:31.697780 14834 catalog_manager.cc:1347] Generated new cluster ID: 6afe1b6a974641558e5980f329ddb350
I20260923 23:18:31.697846 14834 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 23:18:31.703692 14834 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 23:18:31.703840 14834 catalog_manager.cc:1504] Loading token signing keys...
I20260923 23:18:31.718029 14834 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4a7505d9b24d4c6d8c34a0ed87bdbbfe: Generated new TSK 0
I20260923 23:18:31.718194 14834 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 23:18:31.721751 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 14747
2026-09-23T23:18:31Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1247 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790205259943518-25487-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 23:18:31 dist-test-slave-754n krb5kdc[14846](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 23:18:31 dist-test-slave-754n krb5kdc[14846](info): set up 2 sockets
Sep 23 23:18:31 dist-test-slave-754n krb5kdc[14846](info): commencing operation
krb5kdc: starting...
W20260923 23:18:32.723858 25487 mini_kdc.cc:121] Time spent starting KDC: real 0.985s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790205259943518-25487-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790205259943518-25487-0/krb5kdc/test-user.keytab.
Sep 23 23:18:32 dist-test-slave-754n krb5kdc[14846](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205512, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T23:18:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T23:18:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.227.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.227.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.227.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab.
Entry for principal kudu/127.24.227.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab.
I20260923 23:18:32.831760 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790205259943518-25487-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790205259943518-25487-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790205259943518-25487-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.254:40967
--webserver_interface=127.24.227.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab
--principal=kudu/127.24.227.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:40931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.227.254:40967
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:18:32.908870 14859 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:18:32.909046 14859 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:18:32.909067 14859 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:18:32.910226 14859 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 23:18:32.910269 14859 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:18:32.910283 14859 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:18:32.910363 14859 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 23:18:32.910391 14859 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 23:18:32.911836 14859 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:40931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790205259943518-25487-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790205259943518-25487-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.227.254:40967
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab
--principal=kudu/127.24.227.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.254:40967
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.227.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14859
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790205259943518-25487-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:18:32.912099 14859 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:18:32.912321 14859 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:18:32.916831 14865 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:18:32.916836 14864 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:18:32.916846 14867 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:18:32.917047 14859 server_base.cc:1034] running on GCE node
I20260923 23:18:32.917289 14859 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:18:32.917549 14859 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:18:32.918712 14859 hybrid_clock.cc:648] HybridClock initialized: now 1790205512918688 us; error 38 us; skew 500 ppm
Sep 23 23:18:32 dist-test-slave-754n krb5kdc[14846](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205512, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:18:32.921036 14859 init.cc:377] Logged in from keytab as kudu/127.24.227.254@KRBTEST.COM (short username kudu)
I20260923 23:18:32.921396 14859 webserver.cc:460] Webserver started at http://127.24.227.254:44343/ using document root <none> and password file <none>
I20260923 23:18:32.921602 14859 fs_manager.cc:359] Metadata directory not provided
I20260923 23:18:32.921661 14859 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:18:32.921797 14859 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:18:32.922771 14859 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790205259943518-25487-0/minicluster-data/master-0/data/instance:
uuid: "4e69ff5da33e49d99029657eb5dece6f"
format_stamp: "Formatted at 2026-09-23 23:18:32 on dist-test-slave-754n"
server_key: "1d0a3c8023cc39656f03d5e6904d3f9e"
server_key_iv: "043be19f2e22c20cf210b98c20173d20"
server_key_version: "encryptionkey@0"
I20260923 23:18:32.923118 14859 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790205259943518-25487-0/minicluster-data/master-0/wal/instance:
uuid: "4e69ff5da33e49d99029657eb5dece6f"
format_stamp: "Formatted at 2026-09-23 23:18:32 on dist-test-slave-754n"
server_key: "1d0a3c8023cc39656f03d5e6904d3f9e"
server_key_iv: "043be19f2e22c20cf210b98c20173d20"
server_key_version: "encryptionkey@0"
I20260923 23:18:32.924474 14859 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 23:18:32.925202 14874 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:18:32.925369 14859 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 23:18:32.925460 14859 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790205259943518-25487-0/minicluster-data/master-0/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790205259943518-25487-0/minicluster-data/master-0/wal
uuid: "4e69ff5da33e49d99029657eb5dece6f"
format_stamp: "Formatted at 2026-09-23 23:18:32 on dist-test-slave-754n"
server_key: "1d0a3c8023cc39656f03d5e6904d3f9e"
server_key_iv: "043be19f2e22c20cf210b98c20173d20"
server_key_version: "encryptionkey@0"
I20260923 23:18:32.925536 14859 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790205259943518-25487-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790205259943518-25487-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790205259943518-25487-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:18:32.939525 14859 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:18:32.941380 14859 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:18:32.941533 14859 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:18:32.945084 14859 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.254:40967
I20260923 23:18:32.945118 14926 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.254:40967 every 8 connection(s)
I20260923 23:18:32.945509 14859 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
I20260923 23:18:32.946318 14927 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:18:32.948345 14927 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e69ff5da33e49d99029657eb5dece6f: Bootstrap starting.
I20260923 23:18:32.948855 14927 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4e69ff5da33e49d99029657eb5dece6f: Neither blocks nor log segments found. Creating new log.
I20260923 23:18:32.949077 14927 log.cc:824] T 00000000000000000000000000000000 P 4e69ff5da33e49d99029657eb5dece6f: Log is configured to *not* fsync() on all Append() calls
I20260923 23:18:32.949703 14927 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e69ff5da33e49d99029657eb5dece6f: No bootstrap required, opened a new log
I20260923 23:18:32.951172 14927 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4e69ff5da33e49d99029657eb5dece6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e69ff5da33e49d99029657eb5dece6f" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 40967 } }
I20260923 23:18:32.951297 14927 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4e69ff5da33e49d99029657eb5dece6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:18:32.951321 14927 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4e69ff5da33e49d99029657eb5dece6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e69ff5da33e49d99029657eb5dece6f, State: Initialized, Role: FOLLOWER
I20260923 23:18:32.951447 14927 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4e69ff5da33e49d99029657eb5dece6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e69ff5da33e49d99029657eb5dece6f" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 40967 } }
I20260923 23:18:32.951529 14927 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4e69ff5da33e49d99029657eb5dece6f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 23:18:32.951551 14927 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4e69ff5da33e49d99029657eb5dece6f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 23:18:32.951579 14927 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4e69ff5da33e49d99029657eb5dece6f [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:18:32.952324 14927 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4e69ff5da33e49d99029657eb5dece6f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e69ff5da33e49d99029657eb5dece6f" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 40967 } }
I20260923 23:18:32.952411 14927 leader_election.cc:302] T 00000000000000000000000000000000 P 4e69ff5da33e49d99029657eb5dece6f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4e69ff5da33e49d99029657eb5dece6f; no voters:
I20260923 23:18:32.952574 14927 leader_election.cc:288] T 00000000000000000000000000000000 P 4e69ff5da33e49d99029657eb5dece6f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 23:18:32.952608 14930 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4e69ff5da33e49d99029657eb5dece6f [term 1 FOLLOWER]: Leader election won for term 1
I20260923 23:18:32.952730 14930 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4e69ff5da33e49d99029657eb5dece6f [term 1 LEADER]: Becoming Leader. State: Replica: 4e69ff5da33e49d99029657eb5dece6f, State: Running, Role: LEADER
I20260923 23:18:32.952834 14927 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4e69ff5da33e49d99029657eb5dece6f [sys.catalog]: configured and running, proceeding with master startup.
I20260923 23:18:32.952867 14930 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4e69ff5da33e49d99029657eb5dece6f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e69ff5da33e49d99029657eb5dece6f" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 40967 } }
I20260923 23:18:32.953294 14931 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e69ff5da33e49d99029657eb5dece6f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4e69ff5da33e49d99029657eb5dece6f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e69ff5da33e49d99029657eb5dece6f" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 40967 } } }
I20260923 23:18:32.953500 14931 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e69ff5da33e49d99029657eb5dece6f [sys.catalog]: This master's current role is: LEADER
I20260923 23:18:32.953433 14932 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e69ff5da33e49d99029657eb5dece6f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4e69ff5da33e49d99029657eb5dece6f. Latest consensus state: current_term: 1 leader_uuid: "4e69ff5da33e49d99029657eb5dece6f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e69ff5da33e49d99029657eb5dece6f" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 40967 } } }
I20260923 23:18:32.953671 14932 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e69ff5da33e49d99029657eb5dece6f [sys.catalog]: This master's current role is: LEADER
I20260923 23:18:32.953922 14939 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 23:18:32.954471 14939 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 23:18:32.955610 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 14859
I20260923 23:18:32.955744 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790205259943518-25487-0/minicluster-data/master-0/wal/instance
I20260923 23:18:32.955967 25487 external_mini_cluster.cc:1428] Setting key 372016aa09e6134f4529ffccba6715b4
I20260923 23:18:32.955992 14939 catalog_manager.cc:1347] Generated new cluster ID: b597a9fde7854994aa1353267774e89c
I20260923 23:18:32.956032 14939 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 23:18:32 dist-test-slave-754n krb5kdc[14846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205512, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
I20260923 23:18:32.971158 14950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:32.960430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51593 (local address 127.24.227.254:40967)
0923 23:18:32.960597 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:32.960606 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:32.960637 (+ 31us) server_negotiation.cc:407] Connection header received
0923 23:18:32.960742 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:32.960748 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:32.960805 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:32.960924 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:32.961680 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:32.963058 (+ 1378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:32.964159 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:32.964758 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:32.964868 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:32.967577 (+ 2709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:18:32.967581 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:18:32.967581 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:18:32.967593 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:18:32.969569 (+ 1976us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:32.969956 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:32.969959 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:32.969959 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:32.970064 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:32.970271 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:32.970272 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:32.970272 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:32.970573 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:18:32.970727 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:32.970818 (+ 91us) server_negotiation.cc:299] Negotiation successful
0923 23:18:32.970953 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":84,"threads_started":1}
I20260923 23:18:32.971987 25487 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 23:18:32.980280 14950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:32.973390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44547 (local address 127.24.227.254:40967)
0923 23:18:32.973422 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:32.973423 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:32.973530 (+ 107us) server_negotiation.cc:407] Connection header received
0923 23:18:32.973630 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:32.973630 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:32.973648 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:32.973675 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:32.974298 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:32.975033 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:32.976025 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:32.976340 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:32.976379 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:32.977724 (+ 1345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:18:32.977725 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:18:32.977725 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:18:32.977731 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:18:32.979148 (+ 1417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:32.979495 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:32.979496 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:32.979497 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:32.979540 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:32.979704 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:32.979705 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:32.979705 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:32.979801 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:18:32.979917 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:32.980127 (+ 210us) server_negotiation.cc:299] Negotiation successful
0923 23:18:32.980196 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260923 23:18:32.985472 14939 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 23:18:32.985957 14939 catalog_manager.cc:1504] Loading token signing keys...
I20260923 23:18:32.992486 14939 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4e69ff5da33e49d99029657eb5dece6f: Generated new TSK 0
I20260923 23:18:32.992646 14939 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 23:18:32.995338 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 14859
2026-09-23T23:18:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1273 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790205259943518-25487-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 23:18:33 dist-test-slave-754n krb5kdc[14958](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 23:18:33 dist-test-slave-754n krb5kdc[14958](info): set up 2 sockets
Sep 23 23:18:33 dist-test-slave-754n krb5kdc[14958](info): commencing operation
krb5kdc: starting...
W20260923 23:18:34.007766 25487 mini_kdc.cc:121] Time spent starting KDC: real 0.995s user 0.001s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790205259943518-25487-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790205259943518-25487-0/krb5kdc/test-user.keytab.
Sep 23 23:18:34 dist-test-slave-754n krb5kdc[14958](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205514, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T23:18:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T23:18:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.227.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.227.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.227.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab.
Entry for principal kudu/127.24.227.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab.
I20260923 23:18:34.114775 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790205259943518-25487-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790205259943518-25487-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790205259943518-25487-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.254:39071
--webserver_interface=127.24.227.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab
--principal=kudu/127.24.227.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:44989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.227.254:39071
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:18:34.192379 14971 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:18:34.192581 14971 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:18:34.192603 14971 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:18:34.193717 14971 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 23:18:34.193758 14971 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:18:34.193771 14971 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:18:34.193783 14971 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 23:18:34.193840 14971 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 23:18:34.195187 14971 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:44989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790205259943518-25487-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790205259943518-25487-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.227.254:39071
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab
--principal=kudu/127.24.227.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.254:39071
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.227.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14971
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790205259943518-25487-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:18:34.195413 14971 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:18:34.195617 14971 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:18:34.199923 14979 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:18:34.199924 14976 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:18:34.199930 14977 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:18:34.200301 14971 server_base.cc:1034] running on GCE node
I20260923 23:18:34.200500 14971 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:18:34.200706 14971 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:18:34.201886 14971 hybrid_clock.cc:648] HybridClock initialized: now 1790205514201875 us; error 39 us; skew 500 ppm
Sep 23 23:18:34 dist-test-slave-754n krb5kdc[14958](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205514, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:18:34.204231 14971 init.cc:377] Logged in from keytab as kudu/127.24.227.254@KRBTEST.COM (short username kudu)
I20260923 23:18:34.204665 14971 webserver.cc:460] Webserver started at http://127.24.227.254:41445/ using document root <none> and password file <none>
I20260923 23:18:34.204869 14971 fs_manager.cc:359] Metadata directory not provided
I20260923 23:18:34.204910 14971 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:18:34.205044 14971 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:18:34.205981 14971 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790205259943518-25487-0/minicluster-data/master-0/data/instance:
uuid: "32ec7eeae6114c298e3fdd5f6380be12"
format_stamp: "Formatted at 2026-09-23 23:18:34 on dist-test-slave-754n"
server_key: "04ad9d23b1cc9f1e6176f7555037da14"
server_key_iv: "26fc439e83bbdcb2b12cddcccae51ae7"
server_key_version: "encryptionkey@0"
I20260923 23:18:34.206316 14971 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790205259943518-25487-0/minicluster-data/master-0/wal/instance:
uuid: "32ec7eeae6114c298e3fdd5f6380be12"
format_stamp: "Formatted at 2026-09-23 23:18:34 on dist-test-slave-754n"
server_key: "04ad9d23b1cc9f1e6176f7555037da14"
server_key_iv: "26fc439e83bbdcb2b12cddcccae51ae7"
server_key_version: "encryptionkey@0"
I20260923 23:18:34.207573 14971 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260923 23:18:34.208355 14986 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:18:34.208564 14971 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 23:18:34.208631 14971 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790205259943518-25487-0/minicluster-data/master-0/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790205259943518-25487-0/minicluster-data/master-0/wal
uuid: "32ec7eeae6114c298e3fdd5f6380be12"
format_stamp: "Formatted at 2026-09-23 23:18:34 on dist-test-slave-754n"
server_key: "04ad9d23b1cc9f1e6176f7555037da14"
server_key_iv: "26fc439e83bbdcb2b12cddcccae51ae7"
server_key_version: "encryptionkey@0"
I20260923 23:18:34.208710 14971 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790205259943518-25487-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790205259943518-25487-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790205259943518-25487-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:18:34.229192 14971 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:18:34.231419 14971 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:18:34.231575 14971 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:18:34.235311 14971 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.254:39071
I20260923 23:18:34.235337 15038 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.254:39071 every 8 connection(s)
I20260923 23:18:34.235659 14971 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
I20260923 23:18:34.236440 15039 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:18:34.238445 15039 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32ec7eeae6114c298e3fdd5f6380be12: Bootstrap starting.
I20260923 23:18:34.238718 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 14971
I20260923 23:18:34.238874 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790205259943518-25487-0/minicluster-data/master-0/wal/instance
I20260923 23:18:34.239079 25487 external_mini_cluster.cc:1428] Setting key 2e87b7099be6b5344b5cdd7f7a1df03e
I20260923 23:18:34.239147 15039 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 32ec7eeae6114c298e3fdd5f6380be12: Neither blocks nor log segments found. Creating new log.
I20260923 23:18:34.239495 15039 log.cc:824] T 00000000000000000000000000000000 P 32ec7eeae6114c298e3fdd5f6380be12: Log is configured to *not* fsync() on all Append() calls
I20260923 23:18:34.240199 15039 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32ec7eeae6114c298e3fdd5f6380be12: No bootstrap required, opened a new log
I20260923 23:18:34.241920 15039 raft_consensus.cc:348] T 00000000000000000000000000000000 P 32ec7eeae6114c298e3fdd5f6380be12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32ec7eeae6114c298e3fdd5f6380be12" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 39071 } }
I20260923 23:18:34.242065 15039 raft_consensus.cc:374] T 00000000000000000000000000000000 P 32ec7eeae6114c298e3fdd5f6380be12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:18:34.242111 15039 raft_consensus.cc:729] T 00000000000000000000000000000000 P 32ec7eeae6114c298e3fdd5f6380be12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32ec7eeae6114c298e3fdd5f6380be12, State: Initialized, Role: FOLLOWER
I20260923 23:18:34.242255 15039 consensus_queue.cc:260] T 00000000000000000000000000000000 P 32ec7eeae6114c298e3fdd5f6380be12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32ec7eeae6114c298e3fdd5f6380be12" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 39071 } }
I20260923 23:18:34.242343 15039 raft_consensus.cc:388] T 00000000000000000000000000000000 P 32ec7eeae6114c298e3fdd5f6380be12 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 23:18:34.242386 15039 raft_consensus.cc:482] T 00000000000000000000000000000000 P 32ec7eeae6114c298e3fdd5f6380be12 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 23:18:34.242434 15039 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 32ec7eeae6114c298e3fdd5f6380be12 [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:18:34.242941 15039 raft_consensus.cc:504] T 00000000000000000000000000000000 P 32ec7eeae6114c298e3fdd5f6380be12 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32ec7eeae6114c298e3fdd5f6380be12" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 39071 } }
I20260923 23:18:34.243075 15039 leader_election.cc:302] T 00000000000000000000000000000000 P 32ec7eeae6114c298e3fdd5f6380be12 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 32ec7eeae6114c298e3fdd5f6380be12; no voters:
I20260923 23:18:34.243237 15039 leader_election.cc:288] T 00000000000000000000000000000000 P 32ec7eeae6114c298e3fdd5f6380be12 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 23:18:34.243287 15044 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 32ec7eeae6114c298e3fdd5f6380be12 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 23:18:34.243422 15044 raft_consensus.cc:686] T 00000000000000000000000000000000 P 32ec7eeae6114c298e3fdd5f6380be12 [term 1 LEADER]: Becoming Leader. State: Replica: 32ec7eeae6114c298e3fdd5f6380be12, State: Running, Role: LEADER
I20260923 23:18:34.243482 15039 sys_catalog.cc:564] T 00000000000000000000000000000000 P 32ec7eeae6114c298e3fdd5f6380be12 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 23:18:34.243608 15044 consensus_queue.cc:237] T 00000000000000000000000000000000 P 32ec7eeae6114c298e3fdd5f6380be12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32ec7eeae6114c298e3fdd5f6380be12" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 39071 } }
I20260923 23:18:34.244118 15045 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32ec7eeae6114c298e3fdd5f6380be12 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "32ec7eeae6114c298e3fdd5f6380be12" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32ec7eeae6114c298e3fdd5f6380be12" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 39071 } } }
I20260923 23:18:34.244205 15045 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32ec7eeae6114c298e3fdd5f6380be12 [sys.catalog]: This master's current role is: LEADER
I20260923 23:18:34.244426 15046 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32ec7eeae6114c298e3fdd5f6380be12 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 32ec7eeae6114c298e3fdd5f6380be12. Latest consensus state: current_term: 1 leader_uuid: "32ec7eeae6114c298e3fdd5f6380be12" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32ec7eeae6114c298e3fdd5f6380be12" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 39071 } } }
I20260923 23:18:34.244549 15046 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32ec7eeae6114c298e3fdd5f6380be12 [sys.catalog]: This master's current role is: LEADER
I20260923 23:18:34.244699 15053 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 23:18:34.244760 15042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:34.239571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36069 (local address 127.24.227.254:39071)
0923 23:18:34.239854 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:34.239893 (+ 39us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:34.239924 (+ 31us) server_negotiation.cc:407] Connection header received
0923 23:18:34.240384 (+ 460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:34.240391 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:34.240436 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:34.240593 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:34.240786 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:34.241618 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:34.242511 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:34.242916 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:34.242994 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:34.243692 (+ 698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:18:34.243696 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:18:34.243696 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 23:18:34.243916 (+ 220us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:18:34.243958 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 23:18:34.243973 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:18:34.244039 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:34.244304 (+ 265us) server_negotiation.cc:299] Negotiation successful
0923 23:18:34.244371 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":242,"thread_start_us":122,"threads_started":1}
I20260923 23:18:34.245410 15053 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 23:18:34.245656 25487 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 23:18:34.247192 15053 catalog_manager.cc:1347] Generated new cluster ID: f77d32eed4904d858f9dd3a0de6ec29b
I20260923 23:18:34.247249 15053 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 23:18:34.250419 15042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:34.247103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42731 (local address 127.24.227.254:39071)
0923 23:18:34.247146 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:34.247147 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:34.247316 (+ 169us) server_negotiation.cc:407] Connection header received
0923 23:18:34.247339 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:34.247340 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:34.247349 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:34.247401 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:34.247522 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:34.248204 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:34.249109 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:34.249424 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:34.249498 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:34.250195 (+ 697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:18:34.250196 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:18:34.250197 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 23:18:34.250201 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:18:34.250214 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 23:18:34.250218 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:18:34.250276 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:34.250328 (+ 52us) server_negotiation.cc:299] Negotiation successful
0923 23:18:34.250347 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260923 23:18:34.258952 15053 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 23:18:34.259374 15053 catalog_manager.cc:1504] Loading token signing keys...
I20260923 23:18:34.268342 15053 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 32ec7eeae6114c298e3fdd5f6380be12: Generated new TSK 0
I20260923 23:18:34.268494 15053 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 23:18:34.277652 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 14971
2026-09-23T23:18:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1282 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790205259943518-25487-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 23:18:34 dist-test-slave-754n krb5kdc[15070](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 23:18:34 dist-test-slave-754n krb5kdc[15070](info): set up 2 sockets
Sep 23 23:18:34 dist-test-slave-754n krb5kdc[15070](info): commencing operation
krb5kdc: starting...
W20260923 23:18:35.279804 25487 mini_kdc.cc:121] Time spent starting KDC: real 0.985s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790205259943518-25487-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790205259943518-25487-0/krb5kdc/test-user.keytab.
Sep 23 23:18:35 dist-test-slave-754n krb5kdc[15070](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205515, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T23:18:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T23:18:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.227.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.227.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.227.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab.
Entry for principal kudu/127.24.227.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab.
I20260923 23:18:35.386793 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790205259943518-25487-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790205259943518-25487-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790205259943518-25487-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.254:38059
--webserver_interface=127.24.227.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab
--principal=kudu/127.24.227.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:43865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.227.254:38059
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:18:35.466502 15083 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:18:35.466684 15083 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:18:35.466707 15083 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:18:35.467759 15083 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 23:18:35.467793 15083 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:18:35.467808 15083 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:18:35.467820 15083 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 23:18:35.467832 15083 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 23:18:35.469275 15083 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:43865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790205259943518-25487-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790205259943518-25487-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.227.254:38059
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab
--principal=kudu/127.24.227.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.254:38059
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.227.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15083
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790205259943518-25487-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:18:35.469444 15083 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:18:35.469590 15083 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:18:35.474058 15091 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:18:35.474092 15088 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:18:35.474169 15083 server_base.cc:1034] running on GCE node
W20260923 23:18:35.474058 15089 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:18:35.474555 15083 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:18:35.474834 15083 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:18:35.476006 15083 hybrid_clock.cc:648] HybridClock initialized: now 1790205515475993 us; error 30 us; skew 500 ppm
Sep 23 23:18:35 dist-test-slave-754n krb5kdc[15070](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205515, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:18:35.482664 15083 init.cc:377] Logged in from keytab as kudu/127.24.227.254@KRBTEST.COM (short username kudu)
I20260923 23:18:35.483187 15083 webserver.cc:460] Webserver started at http://127.24.227.254:44097/ using document root <none> and password file <none>
I20260923 23:18:35.483448 15083 fs_manager.cc:359] Metadata directory not provided
I20260923 23:18:35.483515 15083 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:18:35.483644 15083 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:18:35.484936 15083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790205259943518-25487-0/minicluster-data/master-0/data/instance:
uuid: "c4ac9cffedf544979d4b6d09da1aa600"
format_stamp: "Formatted at 2026-09-23 23:18:35 on dist-test-slave-754n"
server_key: "9998df057e00dd8537b1fb5b2d738e07"
server_key_iv: "f4cd61250621cb95938e07e0c31d4daa"
server_key_version: "encryptionkey@0"
I20260923 23:18:35.485327 15083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790205259943518-25487-0/minicluster-data/master-0/wal/instance:
uuid: "c4ac9cffedf544979d4b6d09da1aa600"
format_stamp: "Formatted at 2026-09-23 23:18:35 on dist-test-slave-754n"
server_key: "9998df057e00dd8537b1fb5b2d738e07"
server_key_iv: "f4cd61250621cb95938e07e0c31d4daa"
server_key_version: "encryptionkey@0"
I20260923 23:18:35.486619 15083 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 23:18:35.487278 15098 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:18:35.487416 15083 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 23:18:35.487504 15083 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790205259943518-25487-0/minicluster-data/master-0/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790205259943518-25487-0/minicluster-data/master-0/wal
uuid: "c4ac9cffedf544979d4b6d09da1aa600"
format_stamp: "Formatted at 2026-09-23 23:18:35 on dist-test-slave-754n"
server_key: "9998df057e00dd8537b1fb5b2d738e07"
server_key_iv: "f4cd61250621cb95938e07e0c31d4daa"
server_key_version: "encryptionkey@0"
I20260923 23:18:35.487582 15083 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790205259943518-25487-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790205259943518-25487-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790205259943518-25487-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:18:35.490792 15083 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:18:35.492899 15083 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:18:35.493050 15083 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:18:35.496570 15083 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.254:38059
I20260923 23:18:35.496596 15150 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.254:38059 every 8 connection(s)
I20260923 23:18:35.496857 15083 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
I20260923 23:18:35.497702 15151 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:18:35.499632 15151 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4ac9cffedf544979d4b6d09da1aa600: Bootstrap starting.
I20260923 23:18:35.500257 15151 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c4ac9cffedf544979d4b6d09da1aa600: Neither blocks nor log segments found. Creating new log.
I20260923 23:18:35.500531 15151 log.cc:824] T 00000000000000000000000000000000 P c4ac9cffedf544979d4b6d09da1aa600: Log is configured to *not* fsync() on all Append() calls
I20260923 23:18:35.500911 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 15083
I20260923 23:18:35.501019 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790205259943518-25487-0/minicluster-data/master-0/wal/instance
I20260923 23:18:35.501228 25487 external_mini_cluster.cc:1428] Setting key b3b2f52f542af7af1d9bd1710759a42d
I20260923 23:18:35.501292 15151 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4ac9cffedf544979d4b6d09da1aa600: No bootstrap required, opened a new log
I20260923 23:18:35.502892 15151 raft_consensus.cc:348] T 00000000000000000000000000000000 P c4ac9cffedf544979d4b6d09da1aa600 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4ac9cffedf544979d4b6d09da1aa600" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 38059 } }
I20260923 23:18:35.503036 15151 raft_consensus.cc:374] T 00000000000000000000000000000000 P c4ac9cffedf544979d4b6d09da1aa600 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 23:18:35.503100 15151 raft_consensus.cc:729] T 00000000000000000000000000000000 P c4ac9cffedf544979d4b6d09da1aa600 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4ac9cffedf544979d4b6d09da1aa600, State: Initialized, Role: FOLLOWER
I20260923 23:18:35.503252 15151 consensus_queue.cc:260] T 00000000000000000000000000000000 P c4ac9cffedf544979d4b6d09da1aa600 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4ac9cffedf544979d4b6d09da1aa600" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 38059 } }
I20260923 23:18:35.503351 15151 raft_consensus.cc:388] T 00000000000000000000000000000000 P c4ac9cffedf544979d4b6d09da1aa600 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 23:18:35.503408 15151 raft_consensus.cc:482] T 00000000000000000000000000000000 P c4ac9cffedf544979d4b6d09da1aa600 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 23:18:35.503460 15151 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c4ac9cffedf544979d4b6d09da1aa600 [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:18:35.504110 15155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:35.501647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53271 (local address 127.24.227.254:38059)
0923 23:18:35.501812 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:35.501820 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:35.501851 (+ 31us) server_negotiation.cc:407] Connection header received
0923 23:18:35.502880 (+ 1029us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:35.502886 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:35.502912 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:35.503044 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:35.503688 (+ 644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:18:35.503692 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:18:35.503692 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 23:18:35.503696 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:18:35.503733 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 23:18:35.503752 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:18:35.503826 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:35.503901 (+ 75us) server_negotiation.cc:299] Negotiation successful
0923 23:18:35.503936 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":63,"threads_started":1}
I20260923 23:18:35.504277 15151 raft_consensus.cc:504] T 00000000000000000000000000000000 P c4ac9cffedf544979d4b6d09da1aa600 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4ac9cffedf544979d4b6d09da1aa600" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 38059 } }
I20260923 23:18:35.504393 15151 leader_election.cc:302] T 00000000000000000000000000000000 P c4ac9cffedf544979d4b6d09da1aa600 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c4ac9cffedf544979d4b6d09da1aa600; no voters:
I20260923 23:18:35.504557 15151 leader_election.cc:288] T 00000000000000000000000000000000 P c4ac9cffedf544979d4b6d09da1aa600 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 23:18:35.504582 15156 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c4ac9cffedf544979d4b6d09da1aa600 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 23:18:35.504719 15156 raft_consensus.cc:686] T 00000000000000000000000000000000 P c4ac9cffedf544979d4b6d09da1aa600 [term 1 LEADER]: Becoming Leader. State: Replica: c4ac9cffedf544979d4b6d09da1aa600, State: Running, Role: LEADER
I20260923 23:18:35.504802 25487 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 23:18:35.504809 15151 sys_catalog.cc:564] T 00000000000000000000000000000000 P c4ac9cffedf544979d4b6d09da1aa600 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 23:18:35.504838 15156 consensus_queue.cc:237] T 00000000000000000000000000000000 P c4ac9cffedf544979d4b6d09da1aa600 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4ac9cffedf544979d4b6d09da1aa600" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 38059 } }
I20260923 23:18:35.505223 15156 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4ac9cffedf544979d4b6d09da1aa600 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c4ac9cffedf544979d4b6d09da1aa600. Latest consensus state: current_term: 1 leader_uuid: "c4ac9cffedf544979d4b6d09da1aa600" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4ac9cffedf544979d4b6d09da1aa600" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 38059 } } }
I20260923 23:18:35.505453 15156 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4ac9cffedf544979d4b6d09da1aa600 [sys.catalog]: This master's current role is: LEADER
I20260923 23:18:35.505388 15157 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4ac9cffedf544979d4b6d09da1aa600 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c4ac9cffedf544979d4b6d09da1aa600" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4ac9cffedf544979d4b6d09da1aa600" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 38059 } } }
I20260923 23:18:35.505825 15157 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4ac9cffedf544979d4b6d09da1aa600 [sys.catalog]: This master's current role is: LEADER
I20260923 23:18:35.505987 15165 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 23:18:35.506850 15165 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 23:18:35.507267 15155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:35.506277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50123 (local address 127.24.227.254:38059)
0923 23:18:35.506308 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:35.506310 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:35.506458 (+ 148us) server_negotiation.cc:407] Connection header received
0923 23:18:35.506487 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:35.506488 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:35.506492 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:35.506533 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:35.507049 (+ 516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:18:35.507051 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:18:35.507051 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 23:18:35.507055 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:18:35.507067 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 23:18:35.507071 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:18:35.507096 (+ 25us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:35.507190 (+ 94us) server_negotiation.cc:299] Negotiation successful
0923 23:18:35.507216 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260923 23:18:35.508363 15165 catalog_manager.cc:1347] Generated new cluster ID: 3a31ba70fd684846b1814366500dfb77
I20260923 23:18:35.508410 15165 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 23:18:35.516436 15165 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 23:18:35.516579 15165 catalog_manager.cc:1504] Loading token signing keys...
I20260923 23:18:35.521937 15165 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c4ac9cffedf544979d4b6d09da1aa600: Generated new TSK 0
I20260923 23:18:35.522090 15165 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 23:18:35.524073 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 15083
2026-09-23T23:18:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1245 ms)
[----------] 18 tests from AuthTokenIssuingTest (22721 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790205259943518-25487-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 23:18:35 dist-test-slave-754n krb5kdc[15182](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 23:18:35 dist-test-slave-754n krb5kdc[15182](info): set up 2 sockets
Sep 23 23:18:35 dist-test-slave-754n krb5kdc[15182](info): commencing operation
krb5kdc: starting...
W20260923 23:18:36.531945 25487 mini_kdc.cc:121] Time spent starting KDC: real 0.991s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790205259943518-25487-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790205259943518-25487-0/krb5kdc/test-user.keytab.
Sep 23 23:18:36 dist-test-slave-754n krb5kdc[15182](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205516, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T23:18:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T23:18:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.227.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.227.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.227.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab.
Entry for principal kudu/127.24.227.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab.
I20260923 23:18:36.638639 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790205259943518-25487-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790205259943518-25487-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790205259943518-25487-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.254:37755
--webserver_interface=127.24.227.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab
--principal=kudu/127.24.227.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:43979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.227.254:37755,127.24.227.253:34431,127.24.227.252:35155
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:18:36.714718 15195 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260923 23:18:36.714910 15195 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:18:36.714926 15195 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:18:36.714946 15195 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:18:36.716056 15195 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 23:18:36.716092 15195 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:18:36.716105 15195 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:18:36.716140 15195 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 23:18:36.716169 15195 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 23:18:36.717453 15195 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:43979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790205259943518-25487-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790205259943518-25487-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.227.254:37755,127.24.227.253:34431,127.24.227.252:35155
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab
--principal=kudu/127.24.227.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.254:37755
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.227.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15195
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790205259943518-25487-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:18:36.717623 15195 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:18:36.717831 15195 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:18:36.722411 15200 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:18:36.722441 15201 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:18:36.722859 15203 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:18:36.723204 15195 server_base.cc:1034] running on GCE node
I20260923 23:18:36.723483 15195 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:18:36.723803 15195 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:18:36.724995 15195 hybrid_clock.cc:648] HybridClock initialized: now 1790205516724985 us; error 26 us; skew 500 ppm
Sep 23 23:18:36 dist-test-slave-754n krb5kdc[15182](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205516, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:18:36.727242 15195 init.cc:377] Logged in from keytab as kudu/127.24.227.254@KRBTEST.COM (short username kudu)
I20260923 23:18:36.727671 15195 webserver.cc:460] Webserver started at http://127.24.227.254:40393/ using document root <none> and password file <none>
I20260923 23:18:36.727913 15195 fs_manager.cc:359] Metadata directory not provided
I20260923 23:18:36.727980 15195 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:18:36.728107 15195 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:18:36.729028 15195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790205259943518-25487-0/minicluster-data/master-0/data/instance:
uuid: "949cf9a94d4a419abf4102f5cdf3220b"
format_stamp: "Formatted at 2026-09-23 23:18:36 on dist-test-slave-754n"
server_key: "c3fec891b0d85622db87d54a8373f1ee"
server_key_iv: "bc222653c7553339cc2f83be1c8b6f14"
server_key_version: "encryptionkey@0"
I20260923 23:18:36.729403 15195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790205259943518-25487-0/minicluster-data/master-0/wal/instance:
uuid: "949cf9a94d4a419abf4102f5cdf3220b"
format_stamp: "Formatted at 2026-09-23 23:18:36 on dist-test-slave-754n"
server_key: "c3fec891b0d85622db87d54a8373f1ee"
server_key_iv: "bc222653c7553339cc2f83be1c8b6f14"
server_key_version: "encryptionkey@0"
I20260923 23:18:36.730655 15195 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 23:18:36.731287 15210 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:18:36.731449 15195 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 23:18:36.731542 15195 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790205259943518-25487-0/minicluster-data/master-0/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790205259943518-25487-0/minicluster-data/master-0/wal
uuid: "949cf9a94d4a419abf4102f5cdf3220b"
format_stamp: "Formatted at 2026-09-23 23:18:36 on dist-test-slave-754n"
server_key: "c3fec891b0d85622db87d54a8373f1ee"
server_key_iv: "bc222653c7553339cc2f83be1c8b6f14"
server_key_version: "encryptionkey@0"
I20260923 23:18:36.731624 15195 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790205259943518-25487-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790205259943518-25487-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790205259943518-25487-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:18:36.750168 15195 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:18:36.752363 15195 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:18:36.752521 15195 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:18:36.756402 15195 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.254:37755
I20260923 23:18:36.756443 15262 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.254:37755 every 8 connection(s)
I20260923 23:18:36.756770 15195 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
I20260923 23:18:36.757550 15263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:18:36.759022 15263 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.227.254" port: 37755 } has no permanent_uuid. Determining permanent_uuid...
I20260923 23:18:36.762744 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 15195
I20260923 23:18:36.762827 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790205259943518-25487-0/minicluster-data/master-0/wal/instance
I20260923 23:18:36.763019 25487 external_mini_cluster.cc:1428] Setting key e9d4e2bb9af27c08f1adff60a959dbc4
Sep 23 23:18:36 dist-test-slave-754n krb5kdc[15182](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205516, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.254@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
I20260923 23:18:36.769089 15265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:36.759704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:37755 (local address 127.0.0.1:53608)
0923 23:18:36.759859 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:36.759935 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0923 23:18:36.760419 (+ 484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:18:36.760712 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:18:36.760715 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:18:36.760980 (+ 265us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:18:36.761120 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:36.761136 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:36.762173 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:36.762174 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:36.762876 (+ 702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:36.762880 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:36.763435 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:36.763436 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:36.763502 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:36.764002 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:18:36.764009 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:18:36.765663 (+ 1654us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:18:36.767661 (+ 1998us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:18:36.767667 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:18:36.767668 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:18:36.767863 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:18:36.768170 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:18:36.768170 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:18:36.768171 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:18:36.768262 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:18:36.768695 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:18:36.768699 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:18:36.768755 (+ 56us) client_negotiation.cc:769] Sending connection context
0923 23:18:36.768817 (+ 62us) client_negotiation.cc:240] Negotiation successful
0923 23:18:36.768920 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":62,"threads_started":1}
I20260923 23:18:36.769112 15266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:36.759900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53608 (local address 127.24.227.254:37755)
0923 23:18:36.760036 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:36.760040 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:36.760071 (+ 31us) server_negotiation.cc:407] Connection header received
0923 23:18:36.760581 (+ 510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:36.760583 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:36.760602 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:36.760708 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:36.761248 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:36.762056 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:36.762964 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:36.763341 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:36.763464 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:36.765757 (+ 2293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:18:36.765763 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:18:36.765763 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:18:36.765769 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:18:36.767577 (+ 1808us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:36.767980 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:36.767983 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:36.767984 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:36.768095 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:36.768342 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:36.768343 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:36.768344 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:36.768625 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:18:36.768712 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:36.768907 (+ 195us) server_negotiation.cc:299] Negotiation successful
0923 23:18:36.768991 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":53,"threads_started":1}
I20260923 23:18:36.769920 15263 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.227.253" port: 34431 } has no permanent_uuid. Determining permanent_uuid...
I20260923 23:18:36.770229 15265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:36.770118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.253:34431 (local address 127.0.0.1:42774)
0923 23:18:36.770165 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:36.770211 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.253:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260923 23:18:36.770357 15214 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.227.253:34431: connect: Connection refused (error 111)
I20260923 23:18:36.770705 15265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:36.770620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.253:34431 (local address 127.0.0.1:42776)
0923 23:18:36.770663 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:36.770685 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.253:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260923 23:18:36.770808 15263 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.227.253:34431: Network error: Client connection negotiation failed: client connection to 127.24.227.253:34431: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.24.227.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.227.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.227.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.253.keytab.
Entry for principal kudu/127.24.227.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.253.keytab.
I20260923 23:18:36.786432 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790205259943518-25487-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790205259943518-25487-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790205259943518-25487-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790205259943518-25487-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.253:34431
--webserver_interface=127.24.227.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.253.keytab
--principal=kudu/127.24.227.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:43979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.227.254:37755,127.24.227.253:34431,127.24.227.252:35155
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 23:18:36.820016 15263 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.227.253" port: 34431 } attempt: 1
I20260923 23:18:36.820420 15265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:36.820299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.253:34431 (local address 127.0.0.1:42780)
0923 23:18:36.820359 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:36.820394 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.253:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260923 23:18:36.820767 15265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:36.820675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.253:34431 (local address 127.0.0.1:42788)
0923 23:18:36.820717 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:36.820740 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.253:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260923 23:18:36.820847 15263 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.227.253:34431: Network error: Client connection negotiation failed: client connection to 127.24.227.253:34431: connect: Connection refused (error 111)
W20260923 23:18:36.862187 15270 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260923 23:18:36.862464 15270 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:18:36.862494 15270 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:18:36.862534 15270 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:18:36.863667 15270 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 23:18:36.863730 15270 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:18:36.863755 15270 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:18:36.863785 15270 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 23:18:36.863827 15270 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 23:18:36.865233 15270 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:43979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790205259943518-25487-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790205259943518-25487-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.24.227.254:37755,127.24.227.253:34431,127.24.227.252:35155
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.253.keytab
--principal=kudu/127.24.227.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.253:34431
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790205259943518-25487-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.24.227.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15270
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790205259943518-25487-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:18:36.865512 15270 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:18:36.865720 15270 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:18:36.870211 15276 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:18:36.870249 15275 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:18:36.870249 15270 server_base.cc:1034] running on GCE node
W20260923 23:18:36.870219 15278 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:18:36.870675 15270 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:18:36.870958 15270 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:18:36.872112 15270 hybrid_clock.cc:648] HybridClock initialized: now 1790205516872094 us; error 29 us; skew 500 ppm
Sep 23 23:18:36 dist-test-slave-754n krb5kdc[15182](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205516, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:18:36.874516 15270 init.cc:377] Logged in from keytab as kudu/127.24.227.253@KRBTEST.COM (short username kudu)
I20260923 23:18:36.874866 15270 webserver.cc:460] Webserver started at http://127.24.227.253:41097/ using document root <none> and password file <none>
I20260923 23:18:36.875033 15270 fs_manager.cc:359] Metadata directory not provided
I20260923 23:18:36.875069 15270 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:18:36.875149 15270 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:18:36.876127 15270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790205259943518-25487-0/minicluster-data/master-1/data/instance:
uuid: "20d1d4442d5649f39b2f7c91c46010b7"
format_stamp: "Formatted at 2026-09-23 23:18:36 on dist-test-slave-754n"
server_key: "2835d28f607b16ea0ae5f9852ebb9896"
server_key_iv: "bcc8b07fbe5a6985463a8eb0ec64d9ea"
server_key_version: "encryptionkey@0"
I20260923 23:18:36.876410 15270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790205259943518-25487-0/minicluster-data/master-1/wal/instance:
uuid: "20d1d4442d5649f39b2f7c91c46010b7"
format_stamp: "Formatted at 2026-09-23 23:18:36 on dist-test-slave-754n"
server_key: "2835d28f607b16ea0ae5f9852ebb9896"
server_key_iv: "bcc8b07fbe5a6985463a8eb0ec64d9ea"
server_key_version: "encryptionkey@0"
I20260923 23:18:36.877625 15270 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 23:18:36.878248 15285 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:18:36.878428 15270 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 23:18:36.878509 15270 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790205259943518-25487-0/minicluster-data/master-1/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790205259943518-25487-0/minicluster-data/master-1/wal
uuid: "20d1d4442d5649f39b2f7c91c46010b7"
format_stamp: "Formatted at 2026-09-23 23:18:36 on dist-test-slave-754n"
server_key: "2835d28f607b16ea0ae5f9852ebb9896"
server_key_iv: "bcc8b07fbe5a6985463a8eb0ec64d9ea"
server_key_version: "encryptionkey@0"
I20260923 23:18:36.878558 15270 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790205259943518-25487-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790205259943518-25487-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790205259943518-25487-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:18:36.888972 15263 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.227.253" port: 34431 } attempt: 2
I20260923 23:18:36.889360 15265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:36.889231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.253:34431 (local address 127.0.0.1:42804)
0923 23:18:36.889290 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:36.889327 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.253:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260923 23:18:36.889706 15265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:36.889620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.253:34431 (local address 127.0.0.1:42810)
0923 23:18:36.889668 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:36.889690 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.253:34431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260923 23:18:36.889828 15263 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.227.253:34431: Network error: Client connection negotiation failed: client connection to 127.24.227.253:34431: connect: Connection refused (error 111)
I20260923 23:18:36.901906 15270 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:18:36.904124 15270 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:18:36.904278 15270 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:18:36.908058 15270 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.253:34431
I20260923 23:18:36.908097 15337 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.253:34431 every 8 connection(s)
I20260923 23:18:36.908411 15270 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790205259943518-25487-0/minicluster-data/master-1/data/info.pb
I20260923 23:18:36.909198 15338 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:18:36.910411 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 15270
I20260923 23:18:36.910512 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790205259943518-25487-0/minicluster-data/master-1/wal/instance
I20260923 23:18:36.910689 25487 external_mini_cluster.cc:1428] Setting key 021ff8a54a513cc020cfd3af0491b2bc
I20260923 23:18:36.910943 15338 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.227.254" port: 37755 } has no permanent_uuid. Determining permanent_uuid...
Sep 23 23:18:36 dist-test-slave-754n krb5kdc[15182](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205516, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.253@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.24.227.252@KRBTEST.COM; defaulting to no policy
I20260923 23:18:36.922175 15266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:36.912129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53622 (local address 127.24.227.254:37755)
0923 23:18:36.912198 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:36.912200 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:36.912299 (+ 99us) server_negotiation.cc:407] Connection header received
0923 23:18:36.912849 (+ 550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:36.912851 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:36.912866 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:36.912932 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:36.913853 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:36.914759 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:36.916308 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:36.916704 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:36.916769 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:36.919352 (+ 2583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:18:36.919353 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:18:36.919354 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:18:36.919362 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:18:36.920875 (+ 1513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:36.921247 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:36.921248 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:36.921249 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:36.921311 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:36.921555 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:36.921556 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:36.921557 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:36.921671 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:18:36.921750 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:36.922008 (+ 258us) server_negotiation.cc:299] Negotiation successful
0923 23:18:36.922087 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260923 23:18:36.922358 15340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:36.911763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:37755 (local address 127.0.0.1:53622)
0923 23:18:36.912122 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:36.912196 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0923 23:18:36.912741 (+ 545us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:18:36.913072 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:18:36.913077 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:18:36.913456 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:18:36.913736 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:36.913742 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:36.914872 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:36.914873 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:36.916206 (+ 1333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:36.916209 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:36.916782 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:36.916783 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:36.916913 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:36.917438 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:18:36.917446 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:18:36.919230 (+ 1784us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:18:36.920948 (+ 1718us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:18:36.920954 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:18:36.920956 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:18:36.921166 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:18:36.921378 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:18:36.921379 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:18:36.921380 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:18:36.921488 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:18:36.921745 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:18:36.921751 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:18:36.921882 (+ 131us) client_negotiation.cc:769] Sending connection context
0923 23:18:36.921988 (+ 106us) client_negotiation.cc:240] Negotiation successful
0923 23:18:36.922129 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":91,"threads_started":1}
I20260923 23:18:36.922873 15338 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.227.253" port: 34431 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.227.252@KRBTEST.COM" created.
Sep 23 23:18:36 dist-test-slave-754n krb5kdc[15182](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205516, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.253@KRBTEST.COM for kudu/127.24.227.253@KRBTEST.COM
I20260923 23:18:36.929728 15340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:36.923123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.253:34431 (local address 127.0.0.1:42818)
0923 23:18:36.923172 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:36.923181 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 23:18:36.923224 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:18:36.923500 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:18:36.923501 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:18:36.923628 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:18:36.923668 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:36.923669 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:36.924576 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:36.924577 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:36.925054 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:36.925055 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:36.925536 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:36.925537 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:36.925597 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:36.925991 (+ 394us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:18:36.925995 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:18:36.927054 (+ 1059us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:18:36.928691 (+ 1637us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:18:36.928692 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:18:36.928692 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:18:36.928850 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:18:36.929057 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:18:36.929058 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:18:36.929058 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:18:36.929108 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:18:36.929429 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:18:36.929430 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:18:36.929466 (+ 36us) client_negotiation.cc:769] Sending connection context
0923 23:18:36.929531 (+ 65us) client_negotiation.cc:240] Negotiation successful
0923 23:18:36.929621 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260923 23:18:36.929811 15342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:36.923192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42818 (local address 127.24.227.253:34431)
0923 23:18:36.923328 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:36.923351 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:36.923374 (+ 23us) server_negotiation.cc:407] Connection header received
0923 23:18:36.923394 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:36.923396 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:36.923408 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:36.923481 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:36.923762 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:36.924477 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:36.925145 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:36.925449 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:36.925536 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:36.927146 (+ 1610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:18:36.927152 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:18:36.927153 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:18:36.927158 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:18:36.928591 (+ 1433us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:36.928938 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:36.928940 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:36.928941 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:36.928989 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:36.929218 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:36.929219 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:36.929219 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:36.929351 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:18:36.929436 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:36.929566 (+ 130us) server_negotiation.cc:299] Negotiation successful
0923 23:18:36.929672 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":41,"server-negotiator.queue_time_us":115,"thread_start_us":79,"threads_started":1}
I20260923 23:18:36.930397 15338 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.227.252" port: 35155 } has no permanent_uuid. Determining permanent_uuid...
I20260923 23:18:36.930712 15340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:36.930601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.252:35155 (local address 127.0.0.1:60594)
0923 23:18:36.930645 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:36.930684 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.252:35155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260923 23:18:36.930843 15286 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.227.252:35155: connect: Connection refused (error 111)
I20260923 23:18:36.931291 15340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:36.931204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.252:35155 (local address 127.0.0.1:60606)
0923 23:18:36.931254 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:36.931276 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.252:35155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260923 23:18:36.931397 15338 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.227.252:35155: Network error: Client connection negotiation failed: client connection to 127.24.227.252:35155: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.227.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.252.keytab.
Entry for principal kudu/127.24.227.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.252.keytab.
I20260923 23:18:36.938373 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790205259943518-25487-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790205259943518-25487-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790205259943518-25487-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790205259943518-25487-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.252:35155
--webserver_interface=127.24.227.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.252.keytab
--principal=kudu/127.24.227.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:43979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.227.254:37755,127.24.227.253:34431,127.24.227.252:35155
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 23:18:36.980623 15338 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.227.252" port: 35155 } attempt: 1
I20260923 23:18:36.980996 15263 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.227.253" port: 34431 } attempt: 3
I20260923 23:18:36.981105 15340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:36.980968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.252:35155 (local address 127.0.0.1:60616)
0923 23:18:36.981021 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:36.981058 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.252:35155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260923 23:18:36.981547 15340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:36.981437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.252:35155 (local address 127.0.0.1:60618)
0923 23:18:36.981493 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:36.981524 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.252:35155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260923 23:18:36.981917 15338 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.227.252:35155: Network error: Client connection negotiation failed: client connection to 127.24.227.252:35155: connect: Connection refused (error 111)
Sep 23 23:18:36 dist-test-slave-754n krb5kdc[15182](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205516, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.254@KRBTEST.COM for kudu/127.24.227.253@KRBTEST.COM
I20260923 23:18:36.987483 15342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:36.981311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42822 (local address 127.24.227.253:34431)
0923 23:18:36.981355 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:36.981357 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:36.981397 (+ 40us) server_negotiation.cc:407] Connection header received
0923 23:18:36.981421 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:36.981422 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:36.981431 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:36.981478 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:36.981865 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:36.982394 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:36.983133 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:36.983350 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:36.983434 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:36.985236 (+ 1802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:18:36.985237 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:18:36.985237 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:18:36.985244 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:18:36.986450 (+ 1206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:36.986799 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:36.986800 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:36.986801 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:36.986854 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:36.987034 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:36.987035 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:36.987035 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:36.987135 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:18:36.987205 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:36.987344 (+ 139us) server_negotiation.cc:299] Negotiation successful
0923 23:18:36.987407 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260923 23:18:36.987489 15265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:36.981246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.253:34431 (local address 127.0.0.1:42822)
0923 23:18:36.981293 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:36.981302 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 23:18:36.981353 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:18:36.981506 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:18:36.981508 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:18:36.981678 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:18:36.981786 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:36.981788 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:36.982475 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:36.982476 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:36.983037 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:36.983039 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:36.983435 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:36.983436 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:36.983512 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:36.984001 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:18:36.984006 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:18:36.985146 (+ 1140us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:18:36.986541 (+ 1395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:18:36.986542 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:18:36.986543 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:18:36.986693 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:18:36.986916 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:18:36.986917 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:18:36.986917 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:18:36.986968 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:18:36.987197 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:18:36.987198 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:18:36.987245 (+ 47us) client_negotiation.cc:769] Sending connection context
0923 23:18:36.987323 (+ 78us) client_negotiation.cc:240] Negotiation successful
0923 23:18:36.987408 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260923 23:18:36.987902 15263 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.227.252" port: 35155 } has no permanent_uuid. Determining permanent_uuid...
I20260923 23:18:36.988176 15265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:36.988099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.252:35155 (local address 127.0.0.1:60624)
0923 23:18:36.988134 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:36.988159 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.252:35155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260923 23:18:36.988541 15265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:36.988460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.252:35155 (local address 127.0.0.1:60630)
0923 23:18:36.988502 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:36.988524 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.252:35155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260923 23:18:36.988653 15263 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.227.252:35155: Network error: Client connection negotiation failed: client connection to 127.24.227.252:35155: connect: Connection refused (error 111)
W20260923 23:18:37.016932 15345 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260923 23:18:37.017117 15345 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:18:37.017134 15345 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:18:37.017181 15345 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:18:37.018297 15345 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 23:18:37.018337 15345 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:18:37.018350 15345 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:18:37.018402 15345 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 23:18:37.018425 15345 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 23:18:37.019783 15263 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.227.252" port: 35155 } attempt: 1
I20260923 23:18:37.019817 15345 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:43979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790205259943518-25487-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790205259943518-25487-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.24.227.254:37755,127.24.227.253:34431,127.24.227.252:35155
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.252.keytab
--principal=kudu/127.24.227.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.252:35155
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790205259943518-25487-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.24.227.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15345
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790205259943518-25487-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:18:37.020074 15345 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:18:37.020273 15265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:37.020104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.252:35155 (local address 127.0.0.1:60636)
0923 23:18:37.020191 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:37.020243 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.252:35155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260923 23:18:37.020334 15345 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260923 23:18:37.020641 15265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:37.020531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.252:35155 (local address 127.0.0.1:60642)
0923 23:18:37.020582 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:37.020609 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.252:35155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260923 23:18:37.020735 15263 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.227.252:35155: Network error: Client connection negotiation failed: client connection to 127.24.227.252:35155: connect: Connection refused (error 111)
W20260923 23:18:37.024897 15353 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:18:37.024935 15350 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:18:37.024907 15351 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:18:37.025018 15345 server_base.cc:1034] running on GCE node
I20260923 23:18:37.025300 15345 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:18:37.025619 15345 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:18:37.026780 15345 hybrid_clock.cc:648] HybridClock initialized: now 1790205517026780 us; error 38 us; skew 500 ppm
Sep 23 23:18:37 dist-test-slave-754n krb5kdc[15182](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205517, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:18:37.029089 15345 init.cc:377] Logged in from keytab as kudu/127.24.227.252@KRBTEST.COM (short username kudu)
I20260923 23:18:37.029505 15345 webserver.cc:460] Webserver started at http://127.24.227.252:34863/ using document root <none> and password file <none>
I20260923 23:18:37.029712 15345 fs_manager.cc:359] Metadata directory not provided
I20260923 23:18:37.029783 15345 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:18:37.029928 15345 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:18:37.030902 15345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790205259943518-25487-0/minicluster-data/master-2/data/instance:
uuid: "935b042ca1cd4000b0e3a921a5b627d9"
format_stamp: "Formatted at 2026-09-23 23:18:37 on dist-test-slave-754n"
server_key: "6acdf1135d50fa4186a4451d9a94ecd1"
server_key_iv: "b764c9a06d68167194110ff9f32664fd"
server_key_version: "encryptionkey@0"
I20260923 23:18:37.031236 15345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790205259943518-25487-0/minicluster-data/master-2/wal/instance:
uuid: "935b042ca1cd4000b0e3a921a5b627d9"
format_stamp: "Formatted at 2026-09-23 23:18:37 on dist-test-slave-754n"
server_key: "6acdf1135d50fa4186a4451d9a94ecd1"
server_key_iv: "b764c9a06d68167194110ff9f32664fd"
server_key_version: "encryptionkey@0"
I20260923 23:18:37.032600 15345 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 23:18:37.033270 15360 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:18:37.033444 15345 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 23:18:37.033536 15345 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790205259943518-25487-0/minicluster-data/master-2/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790205259943518-25487-0/minicluster-data/master-2/wal
uuid: "935b042ca1cd4000b0e3a921a5b627d9"
format_stamp: "Formatted at 2026-09-23 23:18:37 on dist-test-slave-754n"
server_key: "6acdf1135d50fa4186a4451d9a94ecd1"
server_key_iv: "b764c9a06d68167194110ff9f32664fd"
server_key_version: "encryptionkey@0"
I20260923 23:18:37.033612 15345 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790205259943518-25487-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790205259943518-25487-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790205259943518-25487-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:18:37.050056 15338 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.227.252" port: 35155 } attempt: 2
I20260923 23:18:37.050431 15340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:37.050305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.252:35155 (local address 127.0.0.1:60648)
0923 23:18:37.050361 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:37.050407 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.252:35155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260923 23:18:37.050738 15340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:37.050643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.252:35155 (local address 127.0.0.1:60664)
0923 23:18:37.050684 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:37.050722 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.252:35155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260923 23:18:37.050813 15338 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.227.252:35155: Network error: Client connection negotiation failed: client connection to 127.24.227.252:35155: connect: Connection refused (error 111)
I20260923 23:18:37.066759 15345 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:18:37.068477 15345 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:18:37.068624 15345 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:18:37.072258 15345 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.252:35155
I20260923 23:18:37.072286 15412 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.252:35155 every 8 connection(s)
I20260923 23:18:37.072616 15345 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790205259943518-25487-0/minicluster-data/master-2/data/info.pb
I20260923 23:18:37.073042 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 15345
I20260923 23:18:37.073156 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790205259943518-25487-0/minicluster-data/master-2/wal/instance
I20260923 23:18:37.073344 25487 external_mini_cluster.cc:1428] Setting key 40e7db39777ad06bac8e6f37b0bec6fb
I20260923 23:18:37.073685 15413 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:18:37.075158 15413 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.227.254" port: 37755 } has no permanent_uuid. Determining permanent_uuid...
Sep 23 23:18:37 dist-test-slave-754n krb5kdc[15182](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205517, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.252@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
Sep 23 23:18:37 dist-test-slave-754n krb5kdc[15182](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205516, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.227.252@KRBTEST.COM
I20260923 23:18:37.083623 15266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:37.075844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53630 (local address 127.24.227.254:37755)
0923 23:18:37.075982 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:37.075983 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:37.076021 (+ 38us) server_negotiation.cc:407] Connection header received
0923 23:18:37.076100 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:37.076101 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:37.076111 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:37.076177 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:37.076660 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:37.077112 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:37.078045 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:37.078311 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:37.078376 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:37.080526 (+ 2150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:18:37.080527 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:18:37.080528 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:18:37.080533 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:18:37.082230 (+ 1697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:37.082712 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:37.082712 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:37.082713 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:37.082766 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:37.083023 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:37.083023 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:37.083023 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:37.083113 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:18:37.083189 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:37.083461 (+ 272us) server_negotiation.cc:299] Negotiation successful
0923 23:18:37.083536 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117}
I20260923 23:18:37.083781 15417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:37.075758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:37755 (local address 127.0.0.1:53630)
0923 23:18:37.075912 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:37.075959 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0923 23:18:37.076034 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:18:37.076214 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:18:37.076216 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:18:37.076427 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:18:37.076558 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:37.076561 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:37.077185 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:37.077186 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:37.077975 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:37.077977 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:37.078382 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:37.078383 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:37.078496 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:37.078969 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:18:37.078976 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:18:37.080449 (+ 1473us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:18:37.082303 (+ 1854us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:18:37.082309 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:18:37.082311 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:18:37.082627 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:18:37.082830 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:18:37.082831 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:18:37.082832 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:18:37.082956 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:18:37.083183 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:18:37.083189 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:18:37.083324 (+ 135us) client_negotiation.cc:769] Sending connection context
0923 23:18:37.083430 (+ 106us) client_negotiation.cc:240] Negotiation successful
0923 23:18:37.083562 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":88,"threads_started":1}
I20260923 23:18:37.084321 15413 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.227.253" port: 34431 } has no permanent_uuid. Determining permanent_uuid...
I20260923 23:18:37.084741 15416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:37.073813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60670 (local address 127.24.227.252:35155)
0923 23:18:37.073993 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:37.074001 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:37.075275 (+ 1274us) server_negotiation.cc:407] Connection header received
0923 23:18:37.075850 (+ 575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:37.075860 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:37.075938 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:37.076066 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:37.076637 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:37.077568 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:37.078305 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:37.078682 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:37.078742 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:37.081115 (+ 2373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:18:37.081122 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:18:37.081123 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:18:37.081130 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:18:37.083088 (+ 1958us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:37.083425 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:37.083427 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:37.083428 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:37.083488 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:37.083897 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:37.083898 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:37.083898 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:37.084069 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:18:37.084188 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:37.084557 (+ 369us) server_negotiation.cc:299] Negotiation successful
0923 23:18:37.084653 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":86,"threads_started":1}
Sep 23 23:18:37 dist-test-slave-754n krb5kdc[15182](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205517, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.252@KRBTEST.COM for kudu/127.24.227.253@KRBTEST.COM
Sep 23 23:18:37 dist-test-slave-754n krb5kdc[15182](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205516, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
I20260923 23:18:37.090934 15417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:37.084528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.253:34431 (local address 127.0.0.1:42830)
0923 23:18:37.084559 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:37.084569 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 23:18:37.084602 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:18:37.084928 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:18:37.084929 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:18:37.085027 (+ 98us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:18:37.085065 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:37.085066 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:37.085767 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:37.085768 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:37.086252 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:37.086253 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:37.086639 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:37.086640 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:37.086704 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:37.087077 (+ 373us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:18:37.087081 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:18:37.088280 (+ 1199us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:18:37.090006 (+ 1726us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:18:37.090007 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:18:37.090007 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:18:37.090200 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:18:37.090411 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:18:37.090412 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:18:37.090412 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:18:37.090476 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:18:37.090749 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:18:37.090749 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:18:37.090778 (+ 29us) client_negotiation.cc:769] Sending connection context
0923 23:18:37.090819 (+ 41us) client_negotiation.cc:240] Negotiation successful
0923 23:18:37.090880 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15}
I20260923 23:18:37.091339 15342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:37.084619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42830 (local address 127.24.227.253:34431)
0923 23:18:37.084678 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:37.084680 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:37.084687 (+ 7us) server_negotiation.cc:407] Connection header received
0923 23:18:37.084715 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:37.084716 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:37.084723 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:37.084795 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:37.085166 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:37.085670 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:37.086335 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:37.086558 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:37.086638 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:37.088371 (+ 1733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:18:37.088372 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:18:37.088372 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:18:37.088378 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:18:37.089909 (+ 1531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:37.090273 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:37.090274 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:37.090274 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:37.090321 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:37.090548 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:37.090549 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:37.090549 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:37.090683 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:18:37.090759 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:37.091177 (+ 418us) server_negotiation.cc:299] Negotiation successful
0923 23:18:37.091259 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260923 23:18:37.091636 15413 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.227.252" port: 35155 } has no permanent_uuid. Determining permanent_uuid...
I20260923 23:18:37.092717 15266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:37.085465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53636 (local address 127.24.227.254:37755)
0923 23:18:37.085504 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:37.085506 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:37.085513 (+ 7us) server_negotiation.cc:407] Connection header received
0923 23:18:37.085532 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:37.085533 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:37.085541 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:37.085586 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:37.086016 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:37.086466 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:37.087073 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:37.087265 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:37.087343 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:37.089552 (+ 2209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:18:37.089553 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:18:37.089553 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:18:37.089560 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:18:37.091171 (+ 1611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:37.091532 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:37.091533 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:37.091534 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:37.091591 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:37.091814 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:37.091815 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:37.091815 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:37.091975 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:18:37.092461 (+ 486us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:37.092501 (+ 40us) server_negotiation.cc:299] Negotiation successful
0923 23:18:37.092610 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
Sep 23 23:18:37 dist-test-slave-754n krb5kdc[15182](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205517, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.252@KRBTEST.COM for kudu/127.24.227.252@KRBTEST.COM
I20260923 23:18:37.095898 15263 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.227.252" port: 35155 } attempt: 2
Sep 23 23:18:37 dist-test-slave-754n krb5kdc[15182](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205516, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.227.253@KRBTEST.COM
I20260923 23:18:37.098570 15417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:37.091839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.252:35155 (local address 127.0.0.1:60678)
0923 23:18:37.091887 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:37.091897 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 23:18:37.091930 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:18:37.092073 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:18:37.092073 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:18:37.092240 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:18:37.092269 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:37.092270 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:37.093005 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:37.093005 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:37.093649 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:37.093652 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:37.094184 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:37.094185 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:37.094229 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:37.094529 (+ 300us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:18:37.094546 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:18:37.095800 (+ 1254us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:18:37.097554 (+ 1754us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:18:37.097555 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:18:37.097555 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:18:37.097753 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:18:37.097932 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:18:37.097933 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:18:37.097934 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:18:37.098014 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:18:37.098272 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:18:37.098273 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:18:37.098303 (+ 30us) client_negotiation.cc:769] Sending connection context
0923 23:18:37.098394 (+ 91us) client_negotiation.cc:240] Negotiation successful
0923 23:18:37.098479 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16}
I20260923 23:18:37.098686 15416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:37.091938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60678 (local address 127.24.227.252:35155)
0923 23:18:37.091970 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:37.091972 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:37.091978 (+ 6us) server_negotiation.cc:407] Connection header received
0923 23:18:37.091993 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:37.091994 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:37.092000 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:37.092045 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:37.092339 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:37.092929 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:37.093736 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:37.094105 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:37.094175 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:37.095919 (+ 1744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:18:37.095920 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:18:37.095921 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:18:37.095927 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:18:37.097473 (+ 1546us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:37.097829 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:37.097830 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:37.097831 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:37.097879 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:37.098071 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:37.098071 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:37.098072 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:37.098191 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:18:37.098255 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:37.098414 (+ 159us) server_negotiation.cc:299] Negotiation successful
0923 23:18:37.098548 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":59,"server-negotiator.queue_time_us":17}
I20260923 23:18:37.099962 15413 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 935b042ca1cd4000b0e3a921a5b627d9: Bootstrap starting.
I20260923 23:18:37.100685 15413 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 935b042ca1cd4000b0e3a921a5b627d9: Neither blocks nor log segments found. Creating new log.
I20260923 23:18:37.101027 15413 log.cc:824] T 00000000000000000000000000000000 P 935b042ca1cd4000b0e3a921a5b627d9: Log is configured to *not* fsync() on all Append() calls
Sep 23 23:18:37 dist-test-slave-754n krb5kdc[15182](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205516, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.254@KRBTEST.COM for kudu/127.24.227.252@KRBTEST.COM
I20260923 23:18:37.101759 15413 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 935b042ca1cd4000b0e3a921a5b627d9: No bootstrap required, opened a new log
I20260923 23:18:37.102304 15342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:37.093445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42846 (local address 127.24.227.253:34431)
0923 23:18:37.093504 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:37.093506 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:37.093515 (+ 9us) server_negotiation.cc:407] Connection header received
0923 23:18:37.093567 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:37.093568 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:37.093576 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:37.093628 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:37.094182 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:37.094874 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:37.095768 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:37.096297 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:37.096373 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:37.099306 (+ 2933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:18:37.099307 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:18:37.099307 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:18:37.099314 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:18:37.101096 (+ 1782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:37.101459 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:37.101460 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:37.101460 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:37.101509 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:37.101694 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:37.101695 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:37.101696 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:37.101823 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:18:37.101976 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:37.102138 (+ 162us) server_negotiation.cc:299] Negotiation successful
0923 23:18:37.102203 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260923 23:18:37.103436 25487 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 23:18:37.104004 15413 raft_consensus.cc:348] T 00000000000000000000000000000000 P 935b042ca1cd4000b0e3a921a5b627d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "949cf9a94d4a419abf4102f5cdf3220b" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 37755 } } peers { permanent_uuid: "20d1d4442d5649f39b2f7c91c46010b7" member_type: VOTER last_known_addr { host: "127.24.227.253" port: 34431 } } peers { permanent_uuid: "935b042ca1cd4000b0e3a921a5b627d9" member_type: VOTER last_known_addr { host: "127.24.227.252" port: 35155 } }
I20260923 23:18:37.104148 15413 raft_consensus.cc:729] T 00000000000000000000000000000000 P 935b042ca1cd4000b0e3a921a5b627d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 935b042ca1cd4000b0e3a921a5b627d9, State: Initialized, Role: FOLLOWER
I20260923 23:18:37.104246 15413 consensus_queue.cc:260] T 00000000000000000000000000000000 P 935b042ca1cd4000b0e3a921a5b627d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "949cf9a94d4a419abf4102f5cdf3220b" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 37755 } } peers { permanent_uuid: "20d1d4442d5649f39b2f7c91c46010b7" member_type: VOTER last_known_addr { host: "127.24.227.253" port: 34431 } } peers { permanent_uuid: "935b042ca1cd4000b0e3a921a5b627d9" member_type: VOTER last_known_addr { host: "127.24.227.252" port: 35155 } }
I20260923 23:18:37.104349 15265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:37.096113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.252:35155 (local address 127.0.0.1:60682)
0923 23:18:37.096415 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:37.096424 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 23:18:37.096454 (+ 30us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:18:37.096753 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:18:37.096754 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:18:37.096875 (+ 121us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:18:37.096900 (+ 25us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:37.096901 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:37.098838 (+ 1937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:37.098839 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:37.099460 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:37.099462 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:37.099753 (+ 291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:37.099754 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:37.099803 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:37.100226 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:18:37.100231 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:18:37.101436 (+ 1205us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:18:37.103298 (+ 1862us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:18:37.103299 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:18:37.103300 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:18:37.103497 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:18:37.103707 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:18:37.103707 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:18:37.103708 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:18:37.103762 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:18:37.104073 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:18:37.104074 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:18:37.104112 (+ 38us) client_negotiation.cc:769] Sending connection context
0923 23:18:37.104159 (+ 47us) client_negotiation.cc:240] Negotiation successful
0923 23:18:37.104265 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287}
I20260923 23:18:37.104540 15413 sys_catalog.cc:564] T 00000000000000000000000000000000 P 935b042ca1cd4000b0e3a921a5b627d9 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 23:18:37.104550 15420 sys_catalog.cc:455] T 00000000000000000000000000000000 P 935b042ca1cd4000b0e3a921a5b627d9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "949cf9a94d4a419abf4102f5cdf3220b" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 37755 } } peers { permanent_uuid: "20d1d4442d5649f39b2f7c91c46010b7" member_type: VOTER last_known_addr { host: "127.24.227.253" port: 34431 } } peers { permanent_uuid: "935b042ca1cd4000b0e3a921a5b627d9" member_type: VOTER last_known_addr { host: "127.24.227.252" port: 35155 } } }
I20260923 23:18:37.104678 15420 sys_catalog.cc:458] T 00000000000000000000000000000000 P 935b042ca1cd4000b0e3a921a5b627d9 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 23:18:37.104722 15418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:37.096515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60682 (local address 127.24.227.252:35155)
0923 23:18:37.096640 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:37.096641 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:37.096647 (+ 6us) server_negotiation.cc:407] Connection header received
0923 23:18:37.096663 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:37.096664 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:37.096677 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:37.096746 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:37.096964 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:37.098749 (+ 1785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:37.099524 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:37.099684 (+ 160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:37.099753 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:37.101522 (+ 1769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:18:37.101523 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:18:37.101523 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:18:37.101529 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:18:37.103186 (+ 1657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:37.103582 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:37.103582 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:37.103583 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:37.103634 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:37.103830 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:37.103831 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:37.103831 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:37.103989 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:18:37.104446 (+ 457us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:37.104484 (+ 38us) server_negotiation.cc:299] Negotiation successful
0923 23:18:37.104547 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":79,"threads_started":1}
I20260923 23:18:37.104413 15237 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "949cf9a94d4a419abf4102f5cdf3220b"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53636
I20260923 23:18:37.105931 15263 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 949cf9a94d4a419abf4102f5cdf3220b: Bootstrap starting.
W20260923 23:18:37.106370 15431 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 935b042ca1cd4000b0e3a921a5b627d9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 23:18:37.106436 15431 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 23:18:37.106570 15263 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 949cf9a94d4a419abf4102f5cdf3220b: Neither blocks nor log segments found. Creating new log.
I20260923 23:18:37.106840 15263 log.cc:824] T 00000000000000000000000000000000 P 949cf9a94d4a419abf4102f5cdf3220b: Log is configured to *not* fsync() on all Append() calls
I20260923 23:18:37.107587 15263 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 949cf9a94d4a419abf4102f5cdf3220b: No bootstrap required, opened a new log
I20260923 23:18:37.106362 15345 master_runner.cc:186] Error getting master registration for 127.24.227.254:37755: OK, instance_id { permanent_uuid: "949cf9a94d4a419abf4102f5cdf3220b" instance_seqno: 1790205516755331 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260923 23:18:37.108273 15345 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260923 23:18:37.108831 15263 raft_consensus.cc:348] T 00000000000000000000000000000000 P 949cf9a94d4a419abf4102f5cdf3220b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "949cf9a94d4a419abf4102f5cdf3220b" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 37755 } } peers { permanent_uuid: "20d1d4442d5649f39b2f7c91c46010b7" member_type: VOTER last_known_addr { host: "127.24.227.253" port: 34431 } } peers { permanent_uuid: "935b042ca1cd4000b0e3a921a5b627d9" member_type: VOTER last_known_addr { host: "127.24.227.252" port: 35155 } }
I20260923 23:18:37.108955 15263 raft_consensus.cc:729] T 00000000000000000000000000000000 P 949cf9a94d4a419abf4102f5cdf3220b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 949cf9a94d4a419abf4102f5cdf3220b, State: Initialized, Role: FOLLOWER
I20260923 23:18:37.109021 15263 consensus_queue.cc:260] T 00000000000000000000000000000000 P 949cf9a94d4a419abf4102f5cdf3220b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "949cf9a94d4a419abf4102f5cdf3220b" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 37755 } } peers { permanent_uuid: "20d1d4442d5649f39b2f7c91c46010b7" member_type: VOTER last_known_addr { host: "127.24.227.253" port: 34431 } } peers { permanent_uuid: "935b042ca1cd4000b0e3a921a5b627d9" member_type: VOTER last_known_addr { host: "127.24.227.252" port: 35155 } }
I20260923 23:18:37.109300 15434 sys_catalog.cc:455] T 00000000000000000000000000000000 P 949cf9a94d4a419abf4102f5cdf3220b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "949cf9a94d4a419abf4102f5cdf3220b" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 37755 } } peers { permanent_uuid: "20d1d4442d5649f39b2f7c91c46010b7" member_type: VOTER last_known_addr { host: "127.24.227.253" port: 34431 } } peers { permanent_uuid: "935b042ca1cd4000b0e3a921a5b627d9" member_type: VOTER last_known_addr { host: "127.24.227.252" port: 35155 } } }
I20260923 23:18:37.109350 15263 sys_catalog.cc:564] T 00000000000000000000000000000000 P 949cf9a94d4a419abf4102f5cdf3220b [sys.catalog]: configured and running, proceeding with master startup.
I20260923 23:18:37.109398 15434 sys_catalog.cc:458] T 00000000000000000000000000000000 P 949cf9a94d4a419abf4102f5cdf3220b [sys.catalog]: This master's current role is: FOLLOWER
I20260923 23:18:37.109402 15237 raft_consensus.cc:482] T 00000000000000000000000000000000 P 949cf9a94d4a419abf4102f5cdf3220b [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260923 23:18:37.109512 15237 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 949cf9a94d4a419abf4102f5cdf3220b [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:18:37.110155 15237 raft_consensus.cc:504] T 00000000000000000000000000000000 P 949cf9a94d4a419abf4102f5cdf3220b [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "949cf9a94d4a419abf4102f5cdf3220b" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 37755 } } peers { permanent_uuid: "20d1d4442d5649f39b2f7c91c46010b7" member_type: VOTER last_known_addr { host: "127.24.227.253" port: 34431 } } peers { permanent_uuid: "935b042ca1cd4000b0e3a921a5b627d9" member_type: VOTER last_known_addr { host: "127.24.227.252" port: 35155 } }
I20260923 23:18:37.110420 15237 leader_election.cc:288] T 00000000000000000000000000000000 P 949cf9a94d4a419abf4102f5cdf3220b [CANDIDATE]: Term 1 election: Requested vote from peers 20d1d4442d5649f39b2f7c91c46010b7 (127.24.227.253:34431), 935b042ca1cd4000b0e3a921a5b627d9 (127.24.227.252:35155)
I20260923 23:18:37.111466 15387 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "949cf9a94d4a419abf4102f5cdf3220b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "935b042ca1cd4000b0e3a921a5b627d9"
I20260923 23:18:37.111570 15387 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 935b042ca1cd4000b0e3a921a5b627d9 [term 0 FOLLOWER]: Advancing to term 1
W20260923 23:18:37.112090 15445 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 949cf9a94d4a419abf4102f5cdf3220b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 23:18:37.112228 15445 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 23:18:37.112476 15387 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 935b042ca1cd4000b0e3a921a5b627d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 949cf9a94d4a419abf4102f5cdf3220b in term 1.
I20260923 23:18:37.112697 15211 leader_election.cc:302] T 00000000000000000000000000000000 P 949cf9a94d4a419abf4102f5cdf3220b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 935b042ca1cd4000b0e3a921a5b627d9, 949cf9a94d4a419abf4102f5cdf3220b; no voters:
I20260923 23:18:37.112942 15434 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 949cf9a94d4a419abf4102f5cdf3220b [term 1 FOLLOWER]: Leader election won for term 1
I20260923 23:18:37.113024 15434 raft_consensus.cc:686] T 00000000000000000000000000000000 P 949cf9a94d4a419abf4102f5cdf3220b [term 1 LEADER]: Becoming Leader. State: Replica: 949cf9a94d4a419abf4102f5cdf3220b, State: Running, Role: LEADER
I20260923 23:18:37.113108 15434 consensus_queue.cc:237] T 00000000000000000000000000000000 P 949cf9a94d4a419abf4102f5cdf3220b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "949cf9a94d4a419abf4102f5cdf3220b" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 37755 } } peers { permanent_uuid: "20d1d4442d5649f39b2f7c91c46010b7" member_type: VOTER last_known_addr { host: "127.24.227.253" port: 34431 } } peers { permanent_uuid: "935b042ca1cd4000b0e3a921a5b627d9" member_type: VOTER last_known_addr { host: "127.24.227.252" port: 35155 } }
I20260923 23:18:37.113517 15434 sys_catalog.cc:455] T 00000000000000000000000000000000 P 949cf9a94d4a419abf4102f5cdf3220b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 949cf9a94d4a419abf4102f5cdf3220b. Latest consensus state: current_term: 1 leader_uuid: "949cf9a94d4a419abf4102f5cdf3220b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "949cf9a94d4a419abf4102f5cdf3220b" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 37755 } } peers { permanent_uuid: "20d1d4442d5649f39b2f7c91c46010b7" member_type: VOTER last_known_addr { host: "127.24.227.253" port: 34431 } } peers { permanent_uuid: "935b042ca1cd4000b0e3a921a5b627d9" member_type: VOTER last_known_addr { host: "127.24.227.252" port: 35155 } } }
I20260923 23:18:37.113610 15434 sys_catalog.cc:458] T 00000000000000000000000000000000 P 949cf9a94d4a419abf4102f5cdf3220b [sys.catalog]: This master's current role is: LEADER
I20260923 23:18:37.111712 15195 master_runner.cc:186] Error getting master registration for 127.24.227.253:34431: OK, instance_id { permanent_uuid: "20d1d4442d5649f39b2f7c91c46010b7" instance_seqno: 1790205516906983 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260923 23:18:37.113953 15195 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260923 23:18:37.118088 15455 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 23:18:37.118260 15455 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 23:18:37.120707 15387 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 935b042ca1cd4000b0e3a921a5b627d9 [term 1 FOLLOWER]: Refusing update from remote peer 949cf9a94d4a419abf4102f5cdf3220b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 23:18:37.121166 15434 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 949cf9a94d4a419abf4102f5cdf3220b [LEADER]: Connected to new peer: Peer: permanent_uuid: "935b042ca1cd4000b0e3a921a5b627d9" member_type: VOTER last_known_addr { host: "127.24.227.252" port: 35155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 23:18:37.122388 15420 sys_catalog.cc:455] T 00000000000000000000000000000000 P 935b042ca1cd4000b0e3a921a5b627d9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 949cf9a94d4a419abf4102f5cdf3220b. Latest consensus state: current_term: 1 leader_uuid: "949cf9a94d4a419abf4102f5cdf3220b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "949cf9a94d4a419abf4102f5cdf3220b" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 37755 } } peers { permanent_uuid: "20d1d4442d5649f39b2f7c91c46010b7" member_type: VOTER last_known_addr { host: "127.24.227.253" port: 34431 } } peers { permanent_uuid: "935b042ca1cd4000b0e3a921a5b627d9" member_type: VOTER last_known_addr { host: "127.24.227.252" port: 35155 } } }
I20260923 23:18:37.122591 15420 sys_catalog.cc:458] T 00000000000000000000000000000000 P 935b042ca1cd4000b0e3a921a5b627d9 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 23:18:37.128932 15342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:37.112643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42850 (local address 127.24.227.253:34431)
0923 23:18:37.112695 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:37.112697 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:37.114291 (+ 1594us) server_negotiation.cc:407] Connection header received
0923 23:18:37.114377 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:37.114378 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:37.114387 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:37.114412 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:37.115111 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:37.115742 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:37.116750 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:37.117022 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:37.117058 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:37.125643 (+ 8585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:18:37.125644 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:18:37.125644 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:18:37.125651 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:18:37.127484 (+ 1833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:37.127968 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:37.127969 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:37.127970 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:37.128020 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:37.128292 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:37.128293 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:37.128293 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:37.128416 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:18:37.128488 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:37.128766 (+ 278us) server_negotiation.cc:299] Negotiation successful
0923 23:18:37.128842 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260923 23:18:37.130738 15434 sys_catalog.cc:455] T 00000000000000000000000000000000 P 949cf9a94d4a419abf4102f5cdf3220b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "949cf9a94d4a419abf4102f5cdf3220b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "949cf9a94d4a419abf4102f5cdf3220b" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 37755 } } peers { permanent_uuid: "20d1d4442d5649f39b2f7c91c46010b7" member_type: VOTER last_known_addr { host: "127.24.227.253" port: 34431 } } peers { permanent_uuid: "935b042ca1cd4000b0e3a921a5b627d9" member_type: VOTER last_known_addr { host: "127.24.227.252" port: 35155 } } }
I20260923 23:18:37.130828 15434 sys_catalog.cc:458] T 00000000000000000000000000000000 P 949cf9a94d4a419abf4102f5cdf3220b [sys.catalog]: This master's current role is: LEADER
I20260923 23:18:37.131214 15266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:37.112884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53638 (local address 127.24.227.254:37755)
0923 23:18:37.112915 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:37.112917 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:37.117547 (+ 4630us) server_negotiation.cc:407] Connection header received
0923 23:18:37.117607 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:37.117608 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:37.117619 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:37.117667 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:37.120223 (+ 2556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:37.120863 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:37.122815 (+ 1952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:37.123097 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:37.123162 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:37.125166 (+ 2004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:18:37.125167 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:18:37.125167 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:18:37.125184 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:18:37.128604 (+ 3420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:37.129474 (+ 870us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:37.129487 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:37.129488 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:37.129545 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:37.129851 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:37.129852 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:37.129853 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:37.129997 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:18:37.130401 (+ 404us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:37.131048 (+ 647us) server_negotiation.cc:299] Negotiation successful
0923 23:18:37.131117 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260923 23:18:37.131341 15420 sys_catalog.cc:455] T 00000000000000000000000000000000 P 935b042ca1cd4000b0e3a921a5b627d9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "949cf9a94d4a419abf4102f5cdf3220b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "949cf9a94d4a419abf4102f5cdf3220b" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 37755 } } peers { permanent_uuid: "20d1d4442d5649f39b2f7c91c46010b7" member_type: VOTER last_known_addr { host: "127.24.227.253" port: 34431 } } peers { permanent_uuid: "935b042ca1cd4000b0e3a921a5b627d9" member_type: VOTER last_known_addr { host: "127.24.227.252" port: 35155 } } }
I20260923 23:18:37.131418 15420 sys_catalog.cc:458] T 00000000000000000000000000000000 P 935b042ca1cd4000b0e3a921a5b627d9 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 23:18:37.131534 15457 mvcc.cc:205] Tried to move back new op lower bound from 7332681798123057152 to 7332681798096084992. Current Snapshot: MvccSnapshot[applied={T|T < 7332681798123057152}]
I20260923 23:18:37.131633 15458 mvcc.cc:205] Tried to move back new op lower bound from 7332681798123057152 to 7332681798096084992. Current Snapshot: MvccSnapshot[applied={T|T < 7332681798123057152}]
I20260923 23:18:37.133069 15418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:37.114079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60692 (local address 127.24.227.252:35155)
0923 23:18:37.114126 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:37.114128 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:37.121898 (+ 7770us) server_negotiation.cc:407] Connection header received
0923 23:18:37.122032 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:37.122033 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:37.122044 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:37.122072 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:37.124011 (+ 1939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:37.124689 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:37.125919 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:37.126264 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:37.126302 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:37.129817 (+ 3515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:18:37.129818 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:18:37.129818 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:18:37.129835 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:18:37.131397 (+ 1562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:37.132037 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:37.132038 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:37.132038 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:37.132091 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:37.132379 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:37.132380 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:37.132380 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:37.132512 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:18:37.132597 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:37.132912 (+ 315us) server_negotiation.cc:299] Negotiation successful
0923 23:18:37.132988 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260923 23:18:37.136567 15455 catalog_manager.cc:1347] Generated new cluster ID: 93890edb9d074040977ec222f7d181a9
I20260923 23:18:37.136642 15455 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 23:18:37.141932 15338 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.227.252" port: 35155 } attempt: 3
Sep 23 23:18:37 dist-test-slave-754n krb5kdc[15182](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205516, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.253@KRBTEST.COM for kudu/127.24.227.252@KRBTEST.COM
I20260923 23:18:37.150048 15340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:37.142168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.252:35155 (local address 127.0.0.1:60694)
0923 23:18:37.142203 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:37.142214 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 23:18:37.142253 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:18:37.142406 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:18:37.142407 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:18:37.142554 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:18:37.142592 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:37.142594 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:37.143485 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:37.143486 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:37.144331 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:37.144333 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:37.144799 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:37.144800 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:37.144851 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:37.145246 (+ 395us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:18:37.145251 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:18:37.146736 (+ 1485us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:18:37.148881 (+ 2145us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:18:37.148882 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:18:37.148882 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:18:37.149100 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:18:37.149370 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:18:37.149371 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:18:37.149371 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:18:37.149442 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:18:37.149745 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:18:37.149746 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:18:37.149803 (+ 57us) client_negotiation.cc:769] Sending connection context
0923 23:18:37.149864 (+ 61us) client_negotiation.cc:240] Negotiation successful
0923 23:18:37.149954 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19}
I20260923 23:18:37.150527 15418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:37.142256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60694 (local address 127.24.227.252:35155)
0923 23:18:37.142303 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:37.142305 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:37.142311 (+ 6us) server_negotiation.cc:407] Connection header received
0923 23:18:37.142340 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:37.142341 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:37.142348 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:37.142403 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:37.142666 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:37.143404 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:37.144417 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:37.144711 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:37.144779 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:37.146844 (+ 2065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:18:37.146845 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:18:37.146845 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:18:37.146851 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:18:37.148771 (+ 1920us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:37.149218 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:37.149219 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:37.149219 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:37.149288 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:37.149518 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:37.149519 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:37.149519 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:37.149647 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:18:37.149703 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:37.150369 (+ 666us) server_negotiation.cc:299] Negotiation successful
0923 23:18:37.150442 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260923 23:18:37.152002 15338 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 20d1d4442d5649f39b2f7c91c46010b7: Bootstrap starting.
I20260923 23:18:37.152733 15338 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 20d1d4442d5649f39b2f7c91c46010b7: Neither blocks nor log segments found. Creating new log.
I20260923 23:18:37.153020 15338 log.cc:824] T 00000000000000000000000000000000 P 20d1d4442d5649f39b2f7c91c46010b7: Log is configured to *not* fsync() on all Append() calls
I20260923 23:18:37.164274 15455 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 23:18:37.164470 15338 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 20d1d4442d5649f39b2f7c91c46010b7: No bootstrap required, opened a new log
I20260923 23:18:37.165596 15455 catalog_manager.cc:1504] Loading token signing keys...
I20260923 23:18:37.166556 15338 raft_consensus.cc:348] T 00000000000000000000000000000000 P 20d1d4442d5649f39b2f7c91c46010b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "949cf9a94d4a419abf4102f5cdf3220b" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 37755 } } peers { permanent_uuid: "20d1d4442d5649f39b2f7c91c46010b7" member_type: VOTER last_known_addr { host: "127.24.227.253" port: 34431 } } peers { permanent_uuid: "935b042ca1cd4000b0e3a921a5b627d9" member_type: VOTER last_known_addr { host: "127.24.227.252" port: 35155 } }
I20260923 23:18:37.166689 15338 raft_consensus.cc:729] T 00000000000000000000000000000000 P 20d1d4442d5649f39b2f7c91c46010b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20d1d4442d5649f39b2f7c91c46010b7, State: Initialized, Role: FOLLOWER
I20260923 23:18:37.166783 15338 consensus_queue.cc:260] T 00000000000000000000000000000000 P 20d1d4442d5649f39b2f7c91c46010b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "949cf9a94d4a419abf4102f5cdf3220b" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 37755 } } peers { permanent_uuid: "20d1d4442d5649f39b2f7c91c46010b7" member_type: VOTER last_known_addr { host: "127.24.227.253" port: 34431 } } peers { permanent_uuid: "935b042ca1cd4000b0e3a921a5b627d9" member_type: VOTER last_known_addr { host: "127.24.227.252" port: 35155 } }
I20260923 23:18:37.167018 15464 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20d1d4442d5649f39b2f7c91c46010b7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "949cf9a94d4a419abf4102f5cdf3220b" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 37755 } } peers { permanent_uuid: "20d1d4442d5649f39b2f7c91c46010b7" member_type: VOTER last_known_addr { host: "127.24.227.253" port: 34431 } } peers { permanent_uuid: "935b042ca1cd4000b0e3a921a5b627d9" member_type: VOTER last_known_addr { host: "127.24.227.252" port: 35155 } } }
I20260923 23:18:37.167079 15338 sys_catalog.cc:564] T 00000000000000000000000000000000 P 20d1d4442d5649f39b2f7c91c46010b7 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 23:18:37.167168 15311 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 20d1d4442d5649f39b2f7c91c46010b7 [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:18:37.167989 15311 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 20d1d4442d5649f39b2f7c91c46010b7 [term 1 FOLLOWER]: Refusing update from remote peer 949cf9a94d4a419abf4102f5cdf3220b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 23:18:37.168152 15312 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "949cf9a94d4a419abf4102f5cdf3220b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "20d1d4442d5649f39b2f7c91c46010b7"
I20260923 23:18:37.168592 15451 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 949cf9a94d4a419abf4102f5cdf3220b [LEADER]: Connected to new peer: Peer: permanent_uuid: "20d1d4442d5649f39b2f7c91c46010b7" member_type: VOTER last_known_addr { host: "127.24.227.253" port: 34431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 23:18:37.169261 15312 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 20d1d4442d5649f39b2f7c91c46010b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 949cf9a94d4a419abf4102f5cdf3220b in term 1.
I20260923 23:18:37.169272 15464 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20d1d4442d5649f39b2f7c91c46010b7 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 23:18:37.171020 15464 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20d1d4442d5649f39b2f7c91c46010b7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 949cf9a94d4a419abf4102f5cdf3220b. Latest consensus state: current_term: 1 leader_uuid: "949cf9a94d4a419abf4102f5cdf3220b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "949cf9a94d4a419abf4102f5cdf3220b" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 37755 } } peers { permanent_uuid: "20d1d4442d5649f39b2f7c91c46010b7" member_type: VOTER last_known_addr { host: "127.24.227.253" port: 34431 } } peers { permanent_uuid: "935b042ca1cd4000b0e3a921a5b627d9" member_type: VOTER last_known_addr { host: "127.24.227.252" port: 35155 } } }
I20260923 23:18:37.171089 15464 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20d1d4442d5649f39b2f7c91c46010b7 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 23:18:37.171114 15464 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20d1d4442d5649f39b2f7c91c46010b7 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "949cf9a94d4a419abf4102f5cdf3220b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "949cf9a94d4a419abf4102f5cdf3220b" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 37755 } } peers { permanent_uuid: "20d1d4442d5649f39b2f7c91c46010b7" member_type: VOTER last_known_addr { host: "127.24.227.253" port: 34431 } } peers { permanent_uuid: "935b042ca1cd4000b0e3a921a5b627d9" member_type: VOTER last_known_addr { host: "127.24.227.252" port: 35155 } } }
I20260923 23:18:37.171144 15464 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20d1d4442d5649f39b2f7c91c46010b7 [sys.catalog]: This master's current role is: FOLLOWER
W20260923 23:18:37.172379 15475 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 20d1d4442d5649f39b2f7c91c46010b7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 23:18:37.172428 15475 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 23:18:37.175446 15455 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 949cf9a94d4a419abf4102f5cdf3220b: Generated new TSK 0
I20260923 23:18:37.175594 15455 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 23:18:37.176417 15477 mvcc.cc:205] Tried to move back new op lower bound from 7332681798301220864 to 7332681798096084992. Current Snapshot: MvccSnapshot[applied={T|T < 7332681798123057152}]
I20260923 23:18:37.177181 15451 sys_catalog.cc:455] T 00000000000000000000000000000000 P 949cf9a94d4a419abf4102f5cdf3220b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "949cf9a94d4a419abf4102f5cdf3220b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "949cf9a94d4a419abf4102f5cdf3220b" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 37755 } } peers { permanent_uuid: "20d1d4442d5649f39b2f7c91c46010b7" member_type: VOTER last_known_addr { host: "127.24.227.253" port: 34431 } } peers { permanent_uuid: "935b042ca1cd4000b0e3a921a5b627d9" member_type: VOTER last_known_addr { host: "127.24.227.252" port: 35155 } } }
I20260923 23:18:37.177287 15451 sys_catalog.cc:458] T 00000000000000000000000000000000 P 949cf9a94d4a419abf4102f5cdf3220b [sys.catalog]: This master's current role is: LEADER
W20260923 23:18:37.200289 15342 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 23:18:37.199011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42862 (local address 127.24.227.253:34431)
0923 23:18:37.199050 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:37.199052 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:37.199624 (+ 572us) server_negotiation.cc:407] Connection header received
0923 23:18:37.199715 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:37.199716 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:37.199726 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:37.199755 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:37.200202 (+ 447us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42862: BlockingRecv error: recv got EOF from 127.0.0.1:42862 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260923 23:18:37.200999 15266 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 23:18:37.199352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53644 (local address 127.24.227.254:37755)
0923 23:18:37.199385 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:37.199387 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:37.200504 (+ 1117us) server_negotiation.cc:407] Connection header received
0923 23:18:37.200597 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:37.200598 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:37.200607 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:37.200636 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:37.200948 (+ 312us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53644: BlockingRecv error: recv got EOF from 127.0.0.1:53644 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260923 23:18:37.201992 15418 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 23:18:37.201321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60698 (local address 127.24.227.252:35155)
0923 23:18:37.201358 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:37.201360 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:37.201482 (+ 122us) server_negotiation.cc:407] Connection header received
0923 23:18:37.201572 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:37.201573 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:37.201582 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:37.201611 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:37.201938 (+ 327us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60698: BlockingRecv error: recv got EOF from 127.0.0.1:60698 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260923 23:18:37.204212 15342 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 23:18:37.203457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42864 (local address 127.24.227.253:34431)
0923 23:18:37.203489 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:37.203491 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:37.203668 (+ 177us) server_negotiation.cc:407] Connection header received
0923 23:18:37.203774 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:37.203775 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:37.203791 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:37.203821 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:37.204174 (+ 353us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42864: BlockingRecv error: recv got EOF from 127.0.0.1:42864 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260923 23:18:37.207156 15342 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 23:18:37.206448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42870 (local address 127.24.227.253:34431)
0923 23:18:37.206479 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:37.206481 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:37.206708 (+ 227us) server_negotiation.cc:407] Connection header received
0923 23:18:37.206796 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:37.206797 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:37.206806 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:37.206833 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:37.207116 (+ 283us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42870: BlockingRecv error: recv got EOF from 127.0.0.1:42870 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260923 23:18:37.210887 15342 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 23:18:37.210134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42882 (local address 127.24.227.253:34431)
0923 23:18:37.210174 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:37.210175 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:37.210436 (+ 261us) server_negotiation.cc:407] Connection header received
0923 23:18:37.210526 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:37.210527 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:37.210536 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:37.210563 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:37.210850 (+ 287us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42882: BlockingRecv error: recv got EOF from 127.0.0.1:42882 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260923 23:18:37.216773 15342 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 23:18:37.215986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42886 (local address 127.24.227.253:34431)
0923 23:18:37.216022 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:37.216024 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:37.216281 (+ 257us) server_negotiation.cc:407] Connection header received
0923 23:18:37.216378 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:37.216379 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:37.216389 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:37.216419 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:37.216731 (+ 312us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42886: BlockingRecv error: recv got EOF from 127.0.0.1:42886 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260923 23:18:37.226637 15342 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 23:18:37.225826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42902 (local address 127.24.227.253:34431)
0923 23:18:37.225877 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:37.225879 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:37.226114 (+ 235us) server_negotiation.cc:407] Connection header received
0923 23:18:37.226220 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:37.226221 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:37.226243 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:37.226272 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:37.226592 (+ 320us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42902: BlockingRecv error: recv got EOF from 127.0.0.1:42902 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
W20260923 23:18:37.244573 15342 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 23:18:37.243703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42910 (local address 127.24.227.253:34431)
0923 23:18:37.243739 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:37.243741 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:37.244023 (+ 282us) server_negotiation.cc:407] Connection header received
0923 23:18:37.244149 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:37.244150 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:37.244162 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:37.244191 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:37.244529 (+ 338us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42910: BlockingRecv error: recv got EOF from 127.0.0.1:42910 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260923 23:18:37.278486 15342 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 23:18:37.277746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42922 (local address 127.24.227.253:34431)
0923 23:18:37.277792 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:37.277795 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:37.277946 (+ 151us) server_negotiation.cc:407] Connection header received
0923 23:18:37.277998 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:37.278000 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:37.278011 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:37.278086 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:37.278429 (+ 343us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42922: BlockingRecv error: recv got EOF from 127.0.0.1:42922 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260923 23:18:37.344313 15342 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 23:18:37.343634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42930 (local address 127.24.227.253:34431)
0923 23:18:37.343685 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:37.343687 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:37.343784 (+ 97us) server_negotiation.cc:407] Connection header received
0923 23:18:37.343817 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:37.343818 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:37.343830 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:37.343949 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:37.344252 (+ 303us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42930: BlockingRecv error: recv got EOF from 127.0.0.1:42930 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
W20260923 23:18:37.474118 15342 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 23:18:37.473353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42940 (local address 127.24.227.253:34431)
0923 23:18:37.473437 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:37.473439 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:37.473535 (+ 96us) server_negotiation.cc:407] Connection header received
0923 23:18:37.473633 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:37.473635 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:37.473646 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:37.473721 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:37.474054 (+ 333us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42940: BlockingRecv error: recv got EOF from 127.0.0.1:42940 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260923 23:18:37.731758 15342 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 23:18:37.731176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42952 (local address 127.24.227.253:34431)
0923 23:18:37.731242 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:37.731244 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:37.731299 (+ 55us) server_negotiation.cc:407] Connection header received
0923 23:18:37.731356 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:37.731357 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:37.731366 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:37.731424 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:37.731687 (+ 263us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42952: BlockingRecv error: recv got EOF from 127.0.0.1:42952 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
I20260923 23:18:38.106814 15431 catalog_manager.cc:1259] Loaded cluster ID: 93890edb9d074040977ec222f7d181a9
I20260923 23:18:38.106869 15431 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 935b042ca1cd4000b0e3a921a5b627d9: loading cluster ID for follower catalog manager: success
I20260923 23:18:38.107535 15431 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 935b042ca1cd4000b0e3a921a5b627d9: acquiring CA information for follower catalog manager: success
I20260923 23:18:38.107779 15431 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 935b042ca1cd4000b0e3a921a5b627d9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260923 23:18:38.172827 15475 catalog_manager.cc:1259] Loaded cluster ID: 93890edb9d074040977ec222f7d181a9
I20260923 23:18:38.172894 15475 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 20d1d4442d5649f39b2f7c91c46010b7: loading cluster ID for follower catalog manager: success
I20260923 23:18:38.173642 15475 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 20d1d4442d5649f39b2f7c91c46010b7: acquiring CA information for follower catalog manager: success
I20260923 23:18:38.173884 15475 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 20d1d4442d5649f39b2f7c91c46010b7: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260923 23:18:38.247130 15546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:38.244844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42958 (local address 127.24.227.253:34431)
0923 23:18:38.244964 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:38.244966 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:38.244976 (+ 10us) server_negotiation.cc:407] Connection header received
0923 23:18:38.245036 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:38.245037 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:38.245084 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:38.245153 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 23:18:38.245297 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:38.245776 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:38.246481 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:38.246661 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:38.246709 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:38.246856 (+ 147us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 23:18:38.246958 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 23:18:38.247015 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:38.247051 (+ 36us) server_negotiation.cc:299] Negotiation successful
0923 23:18:38.247068 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":55,"threads_started":1}
I20260923 23:18:38.251288 15552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:38.248885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60702 (local address 127.24.227.252:35155)
0923 23:18:38.248979 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:38.248981 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:38.248990 (+ 9us) server_negotiation.cc:407] Connection header received
0923 23:18:38.249066 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:38.249067 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:38.249117 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:38.249215 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 23:18:38.249304 (+ 89us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:38.249785 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:38.250504 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:38.250717 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:38.250792 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:38.250953 (+ 161us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 23:18:38.251052 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 23:18:38.251137 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:38.251176 (+ 39us) server_negotiation.cc:299] Negotiation successful
0923 23:18:38.251195 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72,"thread_start_us":46,"threads_started":1}
I20260923 23:18:38.255743 15552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:38.252914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60712 (local address 127.24.227.252:35155)
0923 23:18:38.252955 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:38.252958 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:38.253207 (+ 249us) server_negotiation.cc:407] Connection header received
0923 23:18:38.253287 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:38.253288 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:38.253295 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:38.253348 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 23:18:38.253550 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:38.253998 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:38.255090 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:38.255287 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:38.255358 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:38.255496 (+ 138us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 23:18:38.255526 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 23:18:38.255573 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:38.255647 (+ 74us) server_negotiation.cc:299] Negotiation successful
0923 23:18:38.255667 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 23:18:38.256000 15546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:38.252959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42964 (local address 127.24.227.253:34431)
0923 23:18:38.252998 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:38.253000 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:38.253152 (+ 152us) server_negotiation.cc:407] Connection header received
0923 23:18:38.253224 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:38.253225 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:38.253236 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:38.253286 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 23:18:38.253441 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:38.254291 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:38.255287 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:38.255508 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:38.255573 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:38.255715 (+ 142us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 23:18:38.255748 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 23:18:38.255775 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:38.255914 (+ 139us) server_negotiation.cc:299] Negotiation successful
0923 23:18:38.255939 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260923 23:18:38.256376 15559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:38.252931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53652 (local address 127.24.227.254:37755)
0923 23:18:38.253030 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:38.253032 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:38.253072 (+ 40us) server_negotiation.cc:407] Connection header received
0923 23:18:38.253165 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:38.253166 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:38.253219 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:38.253299 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 23:18:38.253589 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:38.254310 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:38.255393 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:38.255702 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:38.255774 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:38.256005 (+ 231us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 23:18:38.256097 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 23:18:38.256156 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:38.256252 (+ 96us) server_negotiation.cc:299] Negotiation successful
0923 23:18:38.256276 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82,"thread_start_us":62,"threads_started":1}
I20260923 23:18:38.257915 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 15195
I20260923 23:18:38.264902 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 15270
I20260923 23:18:38.269786 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 15345
2026-09-23T23:18:38Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2747 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790205259943518-25487-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 23:18:38 dist-test-slave-754n krb5kdc[15563](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 23:18:38 dist-test-slave-754n krb5kdc[15563](info): set up 2 sockets
Sep 23 23:18:38 dist-test-slave-754n krb5kdc[15563](info): commencing operation
krb5kdc: starting...
W20260923 23:18:39.287993 25487 mini_kdc.cc:121] Time spent starting KDC: real 0.999s user 0.005s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790205259943518-25487-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790205259943518-25487-0/krb5kdc/test-user.keytab.
Sep 23 23:18:39 dist-test-slave-754n krb5kdc[15563](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205519, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T23:18:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T23:18:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.227.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.227.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.227.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab.
Entry for principal kudu/127.24.227.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab.
I20260923 23:18:39.392319 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790205259943518-25487-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790205259943518-25487-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790205259943518-25487-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.254:41375
--webserver_interface=127.24.227.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab
--principal=kudu/127.24.227.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:36857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.227.254:41375,127.24.227.253:35927,127.24.227.252:43385
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 23:18:39.470535 15576 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260923 23:18:39.470722 15576 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:18:39.470739 15576 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:18:39.470806 15576 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:18:39.471932 15576 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 23:18:39.471974 15576 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:18:39.471987 15576 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:18:39.472029 15576 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 23:18:39.472055 15576 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 23:18:39.473455 15576 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:36857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790205259943518-25487-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790205259943518-25487-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.227.254:41375,127.24.227.253:35927,127.24.227.252:43385
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.254.keytab
--principal=kudu/127.24.227.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.254:41375
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.227.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15576
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790205259943518-25487-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:18:39.473696 15576 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:18:39.473905 15576 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:18:39.478590 15584 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:18:39.478633 15582 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:18:39.478806 15581 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:18:39.478883 15576 server_base.cc:1034] running on GCE node
I20260923 23:18:39.479118 15576 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:18:39.479398 15576 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:18:39.480557 15576 hybrid_clock.cc:648] HybridClock initialized: now 1790205519480542 us; error 31 us; skew 500 ppm
Sep 23 23:18:39 dist-test-slave-754n krb5kdc[15563](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205519, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:18:39.483553 15576 init.cc:377] Logged in from keytab as kudu/127.24.227.254@KRBTEST.COM (short username kudu)
I20260923 23:18:39.484110 15576 webserver.cc:460] Webserver started at http://127.24.227.254:39417/ using document root <none> and password file <none>
I20260923 23:18:39.484367 15576 fs_manager.cc:359] Metadata directory not provided
I20260923 23:18:39.484452 15576 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:18:39.484575 15576 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:18:39.485764 15576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790205259943518-25487-0/minicluster-data/master-0/data/instance:
uuid: "e8135a3a36784cb6b7337e5488aca184"
format_stamp: "Formatted at 2026-09-23 23:18:39 on dist-test-slave-754n"
server_key: "eeb61559d37284de87223e2ada369923"
server_key_iv: "60a0210b6d4bf0092e252bbbda236d14"
server_key_version: "encryptionkey@0"
I20260923 23:18:39.486130 15576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790205259943518-25487-0/minicluster-data/master-0/wal/instance:
uuid: "e8135a3a36784cb6b7337e5488aca184"
format_stamp: "Formatted at 2026-09-23 23:18:39 on dist-test-slave-754n"
server_key: "eeb61559d37284de87223e2ada369923"
server_key_iv: "60a0210b6d4bf0092e252bbbda236d14"
server_key_version: "encryptionkey@0"
I20260923 23:18:39.487526 15576 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 23:18:39.488294 15591 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:18:39.488433 15576 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 23:18:39.488529 15576 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790205259943518-25487-0/minicluster-data/master-0/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790205259943518-25487-0/minicluster-data/master-0/wal
uuid: "e8135a3a36784cb6b7337e5488aca184"
format_stamp: "Formatted at 2026-09-23 23:18:39 on dist-test-slave-754n"
server_key: "eeb61559d37284de87223e2ada369923"
server_key_iv: "60a0210b6d4bf0092e252bbbda236d14"
server_key_version: "encryptionkey@0"
I20260923 23:18:39.488590 15576 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790205259943518-25487-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790205259943518-25487-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790205259943518-25487-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:18:39.501448 15576 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:18:39.503605 15576 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:18:39.503765 15576 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:18:39.507550 15576 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.254:41375
I20260923 23:18:39.507576 15643 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.254:41375 every 8 connection(s)
I20260923 23:18:39.507929 15576 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790205259943518-25487-0/minicluster-data/master-0/data/info.pb
I20260923 23:18:39.508673 15644 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:18:39.510075 15644 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41375 } has no permanent_uuid. Determining permanent_uuid...
Sep 23 23:18:39 dist-test-slave-754n krb5kdc[15563](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205519, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.254@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
I20260923 23:18:39.516072 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 15576
I20260923 23:18:39.516193 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790205259943518-25487-0/minicluster-data/master-0/wal/instance
I20260923 23:18:39.516389 25487 external_mini_cluster.cc:1428] Setting key c49c3f73f958aef4ad081400f01cb309
I20260923 23:18:39.519029 15647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:39.510676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58870 (local address 127.24.227.254:41375)
0923 23:18:39.510827 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:39.510830 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:39.510880 (+ 50us) server_negotiation.cc:407] Connection header received
0923 23:18:39.511334 (+ 454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:39.511336 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:39.511351 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:39.511426 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:39.511937 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:39.512676 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:39.513480 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:39.513763 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:39.513842 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:39.515618 (+ 1776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:18:39.515624 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:18:39.515624 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:18:39.515630 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:18:39.517573 (+ 1943us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:39.518027 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:39.518029 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:39.518030 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:39.518103 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:39.518349 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:39.518350 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:39.518350 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:39.518534 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:18:39.518588 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:39.518800 (+ 212us) server_negotiation.cc:299] Negotiation successful
0923 23:18:39.518881 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":91,"threads_started":1}
I20260923 23:18:39.519057 15646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:39.510606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41375 (local address 127.0.0.1:58870)
0923 23:18:39.510769 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:39.510806 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0923 23:18:39.511223 (+ 417us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:18:39.511451 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:18:39.511453 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:18:39.511668 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:18:39.511813 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:39.511817 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:39.512750 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:39.512751 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:39.513408 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:39.513410 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:39.513833 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:39.513834 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:39.513888 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:39.514351 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:18:39.514358 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:18:39.515548 (+ 1190us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:18:39.517663 (+ 2115us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:18:39.517669 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:18:39.517671 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:18:39.517930 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:18:39.518172 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:18:39.518173 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:18:39.518173 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:18:39.518283 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:18:39.518594 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:18:39.518600 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:18:39.518664 (+ 64us) client_negotiation.cc:769] Sending connection context
0923 23:18:39.518767 (+ 103us) client_negotiation.cc:240] Negotiation successful
0923 23:18:39.518968 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"mutex_wait_us":63,"thread_start_us":97,"threads_started":1}
I20260923 23:18:39.519778 15644 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.227.253" port: 35927 } has no permanent_uuid. Determining permanent_uuid...
I20260923 23:18:39.520098 15646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:39.520018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.253:35927 (local address 127.0.0.1:55978)
0923 23:18:39.520048 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:39.520083 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.253:35927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
W20260923 23:18:39.520216 15594 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.227.253:35927: connect: Connection refused (error 111)
I20260923 23:18:39.520632 15646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:39.520534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.253:35927 (local address 127.0.0.1:55990)
0923 23:18:39.520578 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:39.520604 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.253:35927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260923 23:18:39.520731 15644 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.227.253:35927: Network error: Client connection negotiation failed: client connection to 127.24.227.253:35927: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.24.227.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.227.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.227.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.253.keytab.
Entry for principal kudu/127.24.227.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.253.keytab.
I20260923 23:18:39.538228 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790205259943518-25487-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790205259943518-25487-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790205259943518-25487-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790205259943518-25487-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.253:35927
--webserver_interface=127.24.227.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.253.keytab
--principal=kudu/127.24.227.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:36857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.227.254:41375,127.24.227.253:35927,127.24.227.252:43385
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 23:18:39.569928 15644 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.227.253" port: 35927 } attempt: 1
I20260923 23:18:39.570338 15646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:39.570211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.253:35927 (local address 127.0.0.1:56002)
0923 23:18:39.570273 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:39.570311 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.253:35927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260923 23:18:39.570658 15646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:39.570568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.253:35927 (local address 127.0.0.1:56012)
0923 23:18:39.570610 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:39.570634 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.253:35927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260923 23:18:39.570749 15644 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.227.253:35927: Network error: Client connection negotiation failed: client connection to 127.24.227.253:35927: connect: Connection refused (error 111)
W20260923 23:18:39.613811 15651 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260923 23:18:39.614024 15651 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:18:39.614054 15651 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:18:39.614094 15651 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:18:39.615223 15651 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 23:18:39.615286 15651 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:18:39.615312 15651 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:18:39.615343 15651 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 23:18:39.615367 15651 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 23:18:39.616791 15651 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:36857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790205259943518-25487-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790205259943518-25487-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.24.227.254:41375,127.24.227.253:35927,127.24.227.252:43385
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.253.keytab
--principal=kudu/127.24.227.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.253:35927
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790205259943518-25487-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.24.227.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15651
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790205259943518-25487-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:18:39.617033 15651 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:18:39.617230 15651 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 23:18:39.621730 15656 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:18:39.621760 15657 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:18:39.621760 15659 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:18:39.621871 15651 server_base.cc:1034] running on GCE node
I20260923 23:18:39.622161 15651 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:18:39.622397 15651 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:18:39.623534 15651 hybrid_clock.cc:648] HybridClock initialized: now 1790205519623523 us; error 26 us; skew 500 ppm
Sep 23 23:18:39 dist-test-slave-754n krb5kdc[15563](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205519, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:18:39.625820 15651 init.cc:377] Logged in from keytab as kudu/127.24.227.253@KRBTEST.COM (short username kudu)
I20260923 23:18:39.626168 15651 webserver.cc:460] Webserver started at http://127.24.227.253:45633/ using document root <none> and password file <none>
I20260923 23:18:39.626332 15651 fs_manager.cc:359] Metadata directory not provided
I20260923 23:18:39.626369 15651 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:18:39.626446 15651 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:18:39.627333 15651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790205259943518-25487-0/minicluster-data/master-1/data/instance:
uuid: "8caa054b89104235a283f84b8ef3f691"
format_stamp: "Formatted at 2026-09-23 23:18:39 on dist-test-slave-754n"
server_key: "2965fea2db1bcbd500125ab3461a7e74"
server_key_iv: "ae690b0323e8999e1af0fdc483e034a2"
server_key_version: "encryptionkey@0"
I20260923 23:18:39.627607 15651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790205259943518-25487-0/minicluster-data/master-1/wal/instance:
uuid: "8caa054b89104235a283f84b8ef3f691"
format_stamp: "Formatted at 2026-09-23 23:18:39 on dist-test-slave-754n"
server_key: "2965fea2db1bcbd500125ab3461a7e74"
server_key_iv: "ae690b0323e8999e1af0fdc483e034a2"
server_key_version: "encryptionkey@0"
I20260923 23:18:39.628871 15651 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260923 23:18:39.629531 15666 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:18:39.629693 15651 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 23:18:39.629765 15651 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790205259943518-25487-0/minicluster-data/master-1/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790205259943518-25487-0/minicluster-data/master-1/wal
uuid: "8caa054b89104235a283f84b8ef3f691"
format_stamp: "Formatted at 2026-09-23 23:18:39 on dist-test-slave-754n"
server_key: "2965fea2db1bcbd500125ab3461a7e74"
server_key_iv: "ae690b0323e8999e1af0fdc483e034a2"
server_key_version: "encryptionkey@0"
I20260923 23:18:39.629853 15651 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790205259943518-25487-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790205259943518-25487-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790205259943518-25487-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:18:39.638878 15644 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.227.253" port: 35927 } attempt: 2
I20260923 23:18:39.639230 15646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:39.639106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.253:35927 (local address 127.0.0.1:56022)
0923 23:18:39.639160 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:39.639202 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.253:35927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260923 23:18:39.639537 15646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:39.639458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.253:35927 (local address 127.0.0.1:56038)
0923 23:18:39.639497 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:39.639520 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.253:35927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260923 23:18:39.639631 15644 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.227.253:35927: Network error: Client connection negotiation failed: client connection to 127.24.227.253:35927: connect: Connection refused (error 111)
I20260923 23:18:39.646508 15651 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:18:39.648137 15651 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:18:39.648299 15651 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:18:39.651846 15651 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.253:35927
I20260923 23:18:39.651957 15718 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.253:35927 every 8 connection(s)
I20260923 23:18:39.652249 15651 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790205259943518-25487-0/minicluster-data/master-1/data/info.pb
I20260923 23:18:39.653053 15719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:18:39.654520 15719 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41375 } has no permanent_uuid. Determining permanent_uuid...
Sep 23 23:18:39 dist-test-slave-754n krb5kdc[15563](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205519, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.253@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
I20260923 23:18:39.662194 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 15651
I20260923 23:18:39.662312 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790205259943518-25487-0/minicluster-data/master-1/wal/instance
I20260923 23:18:39.662438 15647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:39.655159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58880 (local address 127.24.227.254:41375)
0923 23:18:39.655229 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:39.655231 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:39.655418 (+ 187us) server_negotiation.cc:407] Connection header received
0923 23:18:39.655828 (+ 410us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:39.655829 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:39.655838 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:39.655933 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:39.656475 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:39.656932 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:39.657788 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:39.657963 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:39.658009 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:39.659986 (+ 1977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:18:39.659987 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:18:39.659988 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:18:39.659993 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:18:39.661154 (+ 1161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:39.661501 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:39.661502 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:39.661502 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:39.661560 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:39.661782 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:39.661782 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:39.661782 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:39.661885 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:18:39.661931 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:39.662232 (+ 301us) server_negotiation.cc:299] Negotiation successful
0923 23:18:39.662321 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260923 23:18:39.662521 25487 external_mini_cluster.cc:1428] Setting key 034fd488f131e1ff2a3870996c30545e
I20260923 23:18:39.662593 15721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:39.655145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41375 (local address 127.0.0.1:58880)
0923 23:18:39.655310 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:39.655341 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0923 23:18:39.655755 (+ 414us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:18:39.655994 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:18:39.655997 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:18:39.656216 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:18:39.656390 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:39.656394 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:39.657001 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:39.657001 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:39.657718 (+ 717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:39.657719 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:39.658027 (+ 308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:39.658029 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:39.658120 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:39.658613 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:18:39.658621 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:18:39.659826 (+ 1205us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:18:39.661225 (+ 1399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:18:39.661231 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:18:39.661233 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:18:39.661439 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:18:39.661617 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:18:39.661617 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:18:39.661617 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:18:39.661711 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:18:39.661939 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:18:39.661943 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:18:39.662087 (+ 144us) client_negotiation.cc:769] Sending connection context
0923 23:18:39.662212 (+ 125us) client_negotiation.cc:240] Negotiation successful
0923 23:18:39.662383 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":90,"threads_started":1}
I20260923 23:18:39.663072 15719 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.227.253" port: 35927 } has no permanent_uuid. Determining permanent_uuid...
Sep 23 23:18:39 dist-test-slave-754n krb5kdc[15563](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205519, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.253@KRBTEST.COM for kudu/127.24.227.253@KRBTEST.COM
I20260923 23:18:39.669979 15721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:39.663313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.253:35927 (local address 127.0.0.1:56054)
0923 23:18:39.663367 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:39.663375 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 23:18:39.663415 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:18:39.663640 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:18:39.663641 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:18:39.663780 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:18:39.663818 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:39.663819 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:39.664648 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:39.664649 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:39.665106 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:39.665108 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:39.665560 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:39.665561 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:39.665625 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:39.666018 (+ 393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:18:39.666022 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:18:39.667036 (+ 1014us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:18:39.669000 (+ 1964us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:18:39.669001 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:18:39.669001 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:18:39.669165 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:18:39.669415 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:18:39.669415 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:18:39.669416 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:18:39.669463 (+ 47us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:18:39.669777 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:18:39.669777 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:18:39.669809 (+ 32us) client_negotiation.cc:769] Sending connection context
0923 23:18:39.669858 (+ 49us) client_negotiation.cc:240] Negotiation successful
0923 23:18:39.669924 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
WARNING: no policy specified for kudu/127.24.227.252@KRBTEST.COM; defaulting to no policy
I20260923 23:18:39.670114 15722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:39.663375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56054 (local address 127.24.227.253:35927)
0923 23:18:39.663512 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:39.663515 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:39.663539 (+ 24us) server_negotiation.cc:407] Connection header received
0923 23:18:39.663554 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:39.663556 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:39.663565 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:39.663641 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:39.663905 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:39.664556 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:39.665194 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:39.665482 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:39.665552 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:39.667119 (+ 1567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:18:39.667134 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:18:39.667135 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:18:39.667142 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:18:39.668896 (+ 1754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:39.669271 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:39.669275 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:39.669275 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:39.669339 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:39.669526 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:39.669527 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:39.669528 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:39.669708 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:18:39.669797 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:39.669889 (+ 92us) server_negotiation.cc:299] Negotiation successful
0923 23:18:39.669986 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":19,"server-negotiator.queue_time_us":112,"thread_start_us":70,"threads_started":1}
I20260923 23:18:39.670632 15719 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.227.252" port: 43385 } has no permanent_uuid. Determining permanent_uuid...
I20260923 23:18:39.670938 15721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:39.670846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.252:43385 (local address 127.0.0.1:49492)
0923 23:18:39.670887 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:39.670923 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.252:43385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260923 23:18:39.671051 15668 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.227.252:43385: connect: Connection refused (error 111)
I20260923 23:18:39.671440 15721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:39.671355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.252:43385 (local address 127.0.0.1:49508)
0923 23:18:39.671399 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:39.671425 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.252:43385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260923 23:18:39.671528 15719 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.227.252:43385: Network error: Client connection negotiation failed: client connection to 127.24.227.252:43385: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.227.252@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.227.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.252.keytab.
Entry for principal kudu/127.24.227.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.252.keytab.
I20260923 23:18:39.685935 25487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
/tmp/dist-test-taskyHY_dZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790205259943518-25487-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790205259943518-25487-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790205259943518-25487-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790205259943518-25487-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.252:43385
--webserver_interface=127.24.227.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.252.keytab
--principal=kudu/127.24.227.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.227.212:36857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.227.254:41375,127.24.227.253:35927,127.24.227.252:43385
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790205259943518-25487-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790205259943518-25487-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790205259943518-25487-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 23:18:39.720719 15719 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.227.252" port: 43385 } attempt: 1
I20260923 23:18:39.721159 15721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:39.721027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.252:43385 (local address 127.0.0.1:49522)
0923 23:18:39.721099 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:39.721137 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.252:43385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260923 23:18:39.721480 15721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:39.721403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.252:43385 (local address 127.0.0.1:49532)
0923 23:18:39.721444 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:39.721466 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.252:43385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260923 23:18:39.721573 15719 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.227.252:43385: Network error: Client connection negotiation failed: client connection to 127.24.227.252:43385: connect: Connection refused (error 111)
I20260923 23:18:39.730790 15644 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.227.253" port: 35927 } attempt: 3
Sep 23 23:18:39 dist-test-slave-754n krb5kdc[15563](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205519, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.254@KRBTEST.COM for kudu/127.24.227.253@KRBTEST.COM
I20260923 23:18:39.738709 15722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:39.731232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56066 (local address 127.24.227.253:35927)
0923 23:18:39.731291 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:39.731294 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:39.731305 (+ 11us) server_negotiation.cc:407] Connection header received
0923 23:18:39.731368 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:39.731369 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:39.731380 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:39.731447 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:39.731807 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:39.732620 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:39.733870 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:39.734213 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:39.734304 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:39.736431 (+ 2127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:18:39.736433 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:18:39.736433 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:18:39.736438 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:18:39.737784 (+ 1346us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:39.738126 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:39.738126 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:39.738127 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:39.738170 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:39.738346 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:39.738347 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:39.738347 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:39.738433 (+ 86us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:18:39.738478 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:39.738611 (+ 133us) server_negotiation.cc:299] Negotiation successful
0923 23:18:39.738660 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260923 23:18:39.738773 15646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:39.731158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.253:35927 (local address 127.0.0.1:56066)
0923 23:18:39.731219 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:39.731231 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 23:18:39.731302 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:18:39.731453 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:18:39.731454 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:18:39.731635 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:18:39.731711 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:39.731713 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:39.732737 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:39.732739 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:39.733763 (+ 1024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:39.733765 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:39.734319 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:39.734320 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:39.734429 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:39.734987 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:18:39.734992 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:18:39.736359 (+ 1367us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:18:39.737861 (+ 1502us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:18:39.737862 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:18:39.737863 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:18:39.738056 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:18:39.738233 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:18:39.738234 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:18:39.738234 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:18:39.738293 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:18:39.738487 (+ 194us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:18:39.738488 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:18:39.738531 (+ 43us) client_negotiation.cc:769] Sending connection context
0923 23:18:39.738598 (+ 67us) client_negotiation.cc:240] Negotiation successful
0923 23:18:39.738688 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260923 23:18:39.739367 15644 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.227.252" port: 43385 } has no permanent_uuid. Determining permanent_uuid...
I20260923 23:18:39.739639 15646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:39.739563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.252:43385 (local address 127.0.0.1:49542)
0923 23:18:39.739593 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:39.739618 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.252:43385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
I20260923 23:18:39.739950 15646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:39.739883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.252:43385 (local address 127.0.0.1:49550)
0923 23:18:39.739914 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:39.739934 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.252:43385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260923 23:18:39.740072 15644 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.227.252:43385: Network error: Client connection negotiation failed: client connection to 127.24.227.252:43385: connect: Connection refused (error 111)
W20260923 23:18:39.765250 15726 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260923 23:18:39.765419 15726 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 23:18:39.765435 15726 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 23:18:39.765453 15726 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 23:18:39.766579 15726 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 23:18:39.766618 15726 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 23:18:39.766634 15726 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 23:18:39.766670 15726 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 23:18:39.766697 15726 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 23:18:39.768111 15726 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.227.212:36857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790205259943518-25487-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790205259943518-25487-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.24.227.254:41375,127.24.227.253:35927,127.24.227.252:43385
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790205259943518-25487-0/krb5kdc/kudu_127.24.227.252.keytab
--principal=kudu/127.24.227.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.227.252:43385
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790205259943518-25487-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.24.227.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15726
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790205259943518-25487-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 22:15:01 UTC on ccycloud.quasar-sngwrb.root.comops.site
build id 4668915
I20260923 23:18:39.768339 15726 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 23:18:39.768539 15726 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260923 23:18:39.771209 15644 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.227.252" port: 43385 } attempt: 1
I20260923 23:18:39.771615 15646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:39.771462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.252:43385 (local address 127.0.0.1:49554)
0923 23:18:39.771540 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:39.771579 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.252:43385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260923 23:18:39.771976 15646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:39.771824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.252:43385 (local address 127.0.0.1:49564)
0923 23:18:39.771897 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:39.771930 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.252:43385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260923 23:18:39.772094 15644 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.227.252:43385: Network error: Client connection negotiation failed: client connection to 127.24.227.252:43385: connect: Connection refused (error 111)
W20260923 23:18:39.773430 15732 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:18:39.773530 15734 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 23:18:39.773443 15731 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 23:18:39.773684 15726 server_base.cc:1034] running on GCE node
I20260923 23:18:39.773866 15726 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 23:18:39.774102 15726 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 23:18:39.775259 15726 hybrid_clock.cc:648] HybridClock initialized: now 1790205519775246 us; error 27 us; skew 500 ppm
Sep 23 23:18:39 dist-test-slave-754n krb5kdc[15563](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205519, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 23:18:39.777555 15726 init.cc:377] Logged in from keytab as kudu/127.24.227.252@KRBTEST.COM (short username kudu)
I20260923 23:18:39.777920 15726 webserver.cc:460] Webserver started at http://127.24.227.252:36875/ using document root <none> and password file <none>
I20260923 23:18:39.778132 15726 fs_manager.cc:359] Metadata directory not provided
I20260923 23:18:39.778192 15726 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 23:18:39.778340 15726 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 23:18:39.779245 15726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790205259943518-25487-0/minicluster-data/master-2/data/instance:
uuid: "73a1f9bc7f5d4e3f9a9b54e3c79b597b"
format_stamp: "Formatted at 2026-09-23 23:18:39 on dist-test-slave-754n"
server_key: "0e6c8e9b8cbb2d99e4d7a77a3186fa29"
server_key_iv: "4fba332c57941f2bf79468929651eb84"
server_key_version: "encryptionkey@0"
I20260923 23:18:39.779577 15726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790205259943518-25487-0/minicluster-data/master-2/wal/instance:
uuid: "73a1f9bc7f5d4e3f9a9b54e3c79b597b"
format_stamp: "Formatted at 2026-09-23 23:18:39 on dist-test-slave-754n"
server_key: "0e6c8e9b8cbb2d99e4d7a77a3186fa29"
server_key_iv: "4fba332c57941f2bf79468929651eb84"
server_key_version: "encryptionkey@0"
I20260923 23:18:39.780825 15726 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 23:18:39.781450 15741 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 23:18:39.781601 15726 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 23:18:39.781682 15726 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790205259943518-25487-0/minicluster-data/master-2/data,/tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790205259943518-25487-0/minicluster-data/master-2/wal
uuid: "73a1f9bc7f5d4e3f9a9b54e3c79b597b"
format_stamp: "Formatted at 2026-09-23 23:18:39 on dist-test-slave-754n"
server_key: "0e6c8e9b8cbb2d99e4d7a77a3186fa29"
server_key_iv: "4fba332c57941f2bf79468929651eb84"
server_key_version: "encryptionkey@0"
I20260923 23:18:39.781754 15726 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790205259943518-25487-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790205259943518-25487-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790205259943518-25487-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 23:18:39.789686 15719 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.227.252" port: 43385 } attempt: 2
I20260923 23:18:39.790009 15721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:39.789894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.252:43385 (local address 127.0.0.1:49580)
0923 23:18:39.789954 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:39.789986 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.252:43385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260923 23:18:39.790349 15721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:39.790240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.252:43385 (local address 127.0.0.1:49596)
0923 23:18:39.790285 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:39.790309 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.227.252:43385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260923 23:18:39.790438 15719 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.227.252:43385: Network error: Client connection negotiation failed: client connection to 127.24.227.252:43385: connect: Connection refused (error 111)
I20260923 23:18:39.793432 15726 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 23:18:39.795364 15726 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 23:18:39.795504 15726 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 23:18:39.799144 15726 rpc_server.cc:307] RPC server started. Bound to: 127.24.227.252:43385
I20260923 23:18:39.799182 15793 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.227.252:43385 every 8 connection(s)
I20260923 23:18:39.799501 15726 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790205259943518-25487-0/minicluster-data/master-2/data/info.pb
I20260923 23:18:39.799690 25487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu as pid 15726
I20260923 23:18:39.799815 25487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskyHY_dZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790205259943518-25487-0/minicluster-data/master-2/wal/instance
I20260923 23:18:39.800068 25487 external_mini_cluster.cc:1428] Setting key 2446a4b1a69107b3cefd8d501bacd003
I20260923 23:18:39.800488 15794 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 23:18:39.802376 15794 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41375 } has no permanent_uuid. Determining permanent_uuid...
Sep 23 23:18:39 dist-test-slave-754n krb5kdc[15563](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205519, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.227.252@KRBTEST.COM
Sep 23 23:18:39 dist-test-slave-754n krb5kdc[15563](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205519, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.252@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
I20260923 23:18:39.810257 15797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:39.800513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49602 (local address 127.24.227.252:43385)
0923 23:18:39.800655 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:39.800662 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:39.800683 (+ 21us) server_negotiation.cc:407] Connection header received
0923 23:18:39.801036 (+ 353us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:39.801041 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:39.801078 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:39.801176 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:39.801898 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:39.802699 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:39.803824 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:39.804432 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:39.804535 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:39.806941 (+ 2406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:18:39.806947 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:18:39.806947 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:18:39.806952 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:18:39.808745 (+ 1793us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:39.809038 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:39.809041 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:39.809041 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:39.809172 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:39.809378 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:39.809379 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:39.809379 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:39.809578 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:18:39.809743 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:39.809919 (+ 176us) server_negotiation.cc:299] Negotiation successful
0923 23:18:39.810040 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":72,"threads_started":1}
I20260923 23:18:39.810460 15647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:39.803000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58882 (local address 127.24.227.254:41375)
0923 23:18:39.803058 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:39.803060 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:39.803231 (+ 171us) server_negotiation.cc:407] Connection header received
0923 23:18:39.803292 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:39.803293 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:39.803301 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:39.803348 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:39.803822 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:39.804528 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:39.805613 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:39.805775 (+ 162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:39.805819 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:39.807823 (+ 2004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:18:39.807824 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:18:39.807824 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:18:39.807830 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:18:39.809173 (+ 1343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:39.809644 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:39.809645 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:39.809646 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:39.809702 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:39.809898 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:39.809899 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:39.809900 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:39.810023 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:18:39.810094 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:39.810287 (+ 193us) server_negotiation.cc:299] Negotiation successful
0923 23:18:39.810368 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260923 23:18:39.810469 15798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:39.802893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.254:41375 (local address 127.0.0.1:58882)
0923 23:18:39.803120 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:39.803159 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0923 23:18:39.803236 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:18:39.803363 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:18:39.803365 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:18:39.803663 (+ 298us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:18:39.803735 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:39.803737 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:39.804603 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:39.804604 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:39.805509 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:39.805512 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:39.805851 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:39.805852 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:39.805926 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:39.806306 (+ 380us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:18:39.806314 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:18:39.807750 (+ 1436us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:18:39.809383 (+ 1633us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:18:39.809388 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:18:39.809390 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:18:39.809574 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:18:39.809770 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:18:39.809770 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:18:39.809771 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:18:39.809836 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:18:39.810102 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:18:39.810109 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:18:39.810157 (+ 48us) client_negotiation.cc:769] Sending connection context
0923 23:18:39.810238 (+ 81us) client_negotiation.cc:240] Negotiation successful
0923 23:18:39.810334 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":122,"threads_started":1}
I20260923 23:18:39.811040 15794 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.227.253" port: 35927 } has no permanent_uuid. Determining permanent_uuid...
Sep 23 23:18:39 dist-test-slave-754n krb5kdc[15563](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205519, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.252@KRBTEST.COM for kudu/127.24.227.253@KRBTEST.COM
Sep 23 23:18:39 dist-test-slave-754n krb5kdc[15563](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205519, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.227.254@KRBTEST.COM
I20260923 23:18:39.817262 15798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:39.811248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.253:35927 (local address 127.0.0.1:56068)
0923 23:18:39.811292 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:39.811302 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 23:18:39.811346 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:18:39.811457 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:18:39.811457 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:18:39.811572 (+ 115us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:18:39.811615 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:39.811616 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:39.812291 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:39.812292 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:39.812803 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:39.812805 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:39.813244 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:39.813245 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:39.813309 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:39.813711 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:18:39.813716 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:18:39.814846 (+ 1130us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:18:39.816415 (+ 1569us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:18:39.816416 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:18:39.816416 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:18:39.816611 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:18:39.816790 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:18:39.816791 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:18:39.816792 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:18:39.816849 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:18:39.817048 (+ 199us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:18:39.817049 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:18:39.817089 (+ 40us) client_negotiation.cc:769] Sending connection context
0923 23:18:39.817147 (+ 58us) client_negotiation.cc:240] Negotiation successful
0923 23:18:39.817215 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260923 23:18:39.817343 15722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:39.811308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56068 (local address 127.24.227.253:35927)
0923 23:18:39.811359 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:39.811360 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:39.811366 (+ 6us) server_negotiation.cc:407] Connection header received
0923 23:18:39.811389 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:39.811390 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:39.811399 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:39.811454 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:39.811684 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:39.812225 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:39.812879 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:39.813182 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:39.813233 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:39.814943 (+ 1710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:18:39.814944 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:18:39.814945 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:18:39.814950 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:18:39.816344 (+ 1394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:39.816678 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:39.816678 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:39.816678 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:39.816729 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:39.816909 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:39.816910 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:39.816910 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:39.816992 (+ 82us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:18:39.817035 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:39.817185 (+ 150us) server_negotiation.cc:299] Negotiation successful
0923 23:18:39.817249 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260923 23:18:39.817615 15794 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.227.252" port: 43385 } has no permanent_uuid. Determining permanent_uuid...
I20260923 23:18:39.818373 15647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:39.811072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58894 (local address 127.24.227.254:41375)
0923 23:18:39.811105 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:39.811106 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:39.811177 (+ 71us) server_negotiation.cc:407] Connection header received
0923 23:18:39.811254 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:39.811255 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:39.811264 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:39.811292 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:39.811913 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:39.812390 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:39.813071 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:39.813279 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:39.813347 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:39.815476 (+ 2129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:18:39.815477 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:18:39.815478 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:18:39.815484 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:18:39.817030 (+ 1546us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:39.817390 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:39.817391 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:39.817391 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:39.817443 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:39.817752 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:39.817753 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:39.817753 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:39.817866 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:18:39.818043 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:39.818212 (+ 169us) server_negotiation.cc:299] Negotiation successful
0923 23:18:39.818300 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
Sep 23 23:18:39 dist-test-slave-754n krb5kdc[15563](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205519, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.252@KRBTEST.COM for kudu/127.24.227.252@KRBTEST.COM
Sep 23 23:18:39 dist-test-slave-754n krb5kdc[15563](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205519, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.227.253@KRBTEST.COM
I20260923 23:18:39.823416 15798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:39.817817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.252:43385 (local address 127.0.0.1:49608)
0923 23:18:39.817894 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:39.817901 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0923 23:18:39.817952 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:18:39.818055 (+ 103us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:18:39.818056 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:18:39.818144 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:18:39.818165 (+ 21us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:39.818165 (+ 0us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:39.818882 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:39.818883 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:39.819291 (+ 408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:39.819292 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:39.819643 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:39.819643 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:39.819693 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:39.820051 (+ 358us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:18:39.820055 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:18:39.821032 (+ 977us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:18:39.822356 (+ 1324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:18:39.822356 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:18:39.822357 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:18:39.822499 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:18:39.822697 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:18:39.822698 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:18:39.822699 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:18:39.822768 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:18:39.823060 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:18:39.823060 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:18:39.823115 (+ 55us) client_negotiation.cc:769] Sending connection context
0923 23:18:39.823188 (+ 73us) client_negotiation.cc:240] Negotiation successful
0923 23:18:39.823281 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66}
I20260923 23:18:39.823496 15797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:39.817861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49608 (local address 127.24.227.252:43385)
0923 23:18:39.817910 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:39.817912 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:39.817945 (+ 33us) server_negotiation.cc:407] Connection header received
0923 23:18:39.817997 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:39.817997 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:39.818005 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:39.818054 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:39.818232 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:39.818785 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:39.819385 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:39.819570 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:39.819643 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:39.821101 (+ 1458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:18:39.821102 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:18:39.821102 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:18:39.821109 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:18:39.822281 (+ 1172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:39.822568 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:39.822569 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:39.822569 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:39.822620 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:39.822839 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:39.822840 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:39.822840 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:39.822989 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:18:39.823064 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:39.823227 (+ 163us) server_negotiation.cc:299] Negotiation successful
0923 23:18:39.823325 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":41,"server-negotiator.queue_time_us":33}
I20260923 23:18:39.824769 15794 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73a1f9bc7f5d4e3f9a9b54e3c79b597b: Bootstrap starting.
I20260923 23:18:39.825520 15794 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 73a1f9bc7f5d4e3f9a9b54e3c79b597b: Neither blocks nor log segments found. Creating new log.
I20260923 23:18:39.825810 15794 log.cc:824] T 00000000000000000000000000000000 P 73a1f9bc7f5d4e3f9a9b54e3c79b597b: Log is configured to *not* fsync() on all Append() calls
I20260923 23:18:39.825845 15722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:39.818995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56078 (local address 127.24.227.253:35927)
0923 23:18:39.819022 (+ 27us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:39.819024 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:39.819063 (+ 39us) server_negotiation.cc:407] Connection header received
0923 23:18:39.819124 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:39.819125 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:39.819133 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:39.819181 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:39.819579 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:39.820035 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:39.820597 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:39.820788 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:39.820855 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:39.823126 (+ 2271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:18:39.823127 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:18:39.823128 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:18:39.823134 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:18:39.824697 (+ 1563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:39.825095 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:39.825096 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:39.825096 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:39.825150 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:39.825366 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:39.825367 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:39.825368 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:39.825470 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:18:39.825641 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:39.825676 (+ 35us) server_negotiation.cc:299] Negotiation successful
0923 23:18:39.825753 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13}
I20260923 23:18:39.826614 15794 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73a1f9bc7f5d4e3f9a9b54e3c79b597b: No bootstrap required, opened a new log
I20260923 23:18:39.827073 25487 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 23:18:39.828341 15794 raft_consensus.cc:348] T 00000000000000000000000000000000 P 73a1f9bc7f5d4e3f9a9b54e3c79b597b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8135a3a36784cb6b7337e5488aca184" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41375 } } peers { permanent_uuid: "8caa054b89104235a283f84b8ef3f691" member_type: VOTER last_known_addr { host: "127.24.227.253" port: 35927 } } peers { permanent_uuid: "73a1f9bc7f5d4e3f9a9b54e3c79b597b" member_type: VOTER last_known_addr { host: "127.24.227.252" port: 43385 } }
I20260923 23:18:39.828442 15794 raft_consensus.cc:729] T 00000000000000000000000000000000 P 73a1f9bc7f5d4e3f9a9b54e3c79b597b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73a1f9bc7f5d4e3f9a9b54e3c79b597b, State: Initialized, Role: FOLLOWER
I20260923 23:18:39.828516 15794 consensus_queue.cc:260] T 00000000000000000000000000000000 P 73a1f9bc7f5d4e3f9a9b54e3c79b597b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8135a3a36784cb6b7337e5488aca184" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41375 } } peers { permanent_uuid: "8caa054b89104235a283f84b8ef3f691" member_type: VOTER last_known_addr { host: "127.24.227.253" port: 35927 } } peers { permanent_uuid: "73a1f9bc7f5d4e3f9a9b54e3c79b597b" member_type: VOTER last_known_addr { host: "127.24.227.252" port: 43385 } }
I20260923 23:18:39.828794 15794 sys_catalog.cc:564] T 00000000000000000000000000000000 P 73a1f9bc7f5d4e3f9a9b54e3c79b597b [sys.catalog]: configured and running, proceeding with master startup.
I20260923 23:18:39.828760 15800 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73a1f9bc7f5d4e3f9a9b54e3c79b597b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8135a3a36784cb6b7337e5488aca184" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41375 } } peers { permanent_uuid: "8caa054b89104235a283f84b8ef3f691" member_type: VOTER last_known_addr { host: "127.24.227.253" port: 35927 } } peers { permanent_uuid: "73a1f9bc7f5d4e3f9a9b54e3c79b597b" member_type: VOTER last_known_addr { host: "127.24.227.252" port: 43385 } } }
I20260923 23:18:39.828099 15618 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "e8135a3a36784cb6b7337e5488aca184"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58894
I20260923 23:18:39.828866 15800 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73a1f9bc7f5d4e3f9a9b54e3c79b597b [sys.catalog]: This master's current role is: FOLLOWER
W20260923 23:18:39.830555 15811 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 73a1f9bc7f5d4e3f9a9b54e3c79b597b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 23:18:39.830595 15811 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 23:18:39.830910 15726 master_runner.cc:186] Error getting master registration for 127.24.227.254:41375: OK, instance_id { permanent_uuid: "e8135a3a36784cb6b7337e5488aca184" instance_seqno: 1790205519506521 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260923 23:18:39.833711 15726 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260923 23:18:39.847201 15644 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.227.252" port: 43385 } attempt: 2
Sep 23 23:18:39 dist-test-slave-754n krb5kdc[15563](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205519, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.254@KRBTEST.COM for kudu/127.24.227.252@KRBTEST.COM
I20260923 23:18:39.853438 15797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:39.847587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49624 (local address 127.24.227.252:43385)
0923 23:18:39.847622 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:39.847624 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:39.847630 (+ 6us) server_negotiation.cc:407] Connection header received
0923 23:18:39.847648 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:39.847648 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:39.847656 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:39.847716 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:39.848018 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:39.848428 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:39.849085 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:39.849284 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:39.849359 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:39.850965 (+ 1606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:18:39.850966 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:18:39.850967 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:18:39.850973 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:18:39.852314 (+ 1341us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:39.852654 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:39.852655 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:39.852656 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:39.852711 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:39.852906 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:39.852907 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:39.852907 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:39.853039 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:18:39.853134 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:39.853293 (+ 159us) server_negotiation.cc:299] Negotiation successful
0923 23:18:39.853357 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260923 23:18:39.853443 15646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:39.847473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.252:43385 (local address 127.0.0.1:49624)
0923 23:18:39.847532 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:39.847542 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 23:18:39.847584 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:18:39.847721 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:18:39.847722 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:18:39.847905 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:18:39.847942 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:39.847944 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:39.848507 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:39.848508 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:39.848984 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:39.848985 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:39.849356 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:39.849357 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:39.849413 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:39.849822 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:18:39.849827 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:18:39.850876 (+ 1049us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:18:39.852423 (+ 1547us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:18:39.852424 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:18:39.852424 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:18:39.852578 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:18:39.852771 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:18:39.852772 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:18:39.852772 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:18:39.852824 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:18:39.853129 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:18:39.853129 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:18:39.853173 (+ 44us) client_negotiation.cc:769] Sending connection context
0923 23:18:39.853269 (+ 96us) client_negotiation.cc:240] Negotiation successful
0923 23:18:39.853357 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260923 23:18:39.854521 15644 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8135a3a36784cb6b7337e5488aca184: Bootstrap starting.
I20260923 23:18:39.855020 15644 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e8135a3a36784cb6b7337e5488aca184: Neither blocks nor log segments found. Creating new log.
I20260923 23:18:39.855225 15644 log.cc:824] T 00000000000000000000000000000000 P e8135a3a36784cb6b7337e5488aca184: Log is configured to *not* fsync() on all Append() calls
I20260923 23:18:39.855850 15644 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8135a3a36784cb6b7337e5488aca184: No bootstrap required, opened a new log
I20260923 23:18:39.856796 15644 raft_consensus.cc:348] T 00000000000000000000000000000000 P e8135a3a36784cb6b7337e5488aca184 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8135a3a36784cb6b7337e5488aca184" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41375 } } peers { permanent_uuid: "8caa054b89104235a283f84b8ef3f691" member_type: VOTER last_known_addr { host: "127.24.227.253" port: 35927 } } peers { permanent_uuid: "73a1f9bc7f5d4e3f9a9b54e3c79b597b" member_type: VOTER last_known_addr { host: "127.24.227.252" port: 43385 } }
I20260923 23:18:39.856885 15644 raft_consensus.cc:729] T 00000000000000000000000000000000 P e8135a3a36784cb6b7337e5488aca184 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8135a3a36784cb6b7337e5488aca184, State: Initialized, Role: FOLLOWER
I20260923 23:18:39.856935 15644 consensus_queue.cc:260] T 00000000000000000000000000000000 P e8135a3a36784cb6b7337e5488aca184 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8135a3a36784cb6b7337e5488aca184" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41375 } } peers { permanent_uuid: "8caa054b89104235a283f84b8ef3f691" member_type: VOTER last_known_addr { host: "127.24.227.253" port: 35927 } } peers { permanent_uuid: "73a1f9bc7f5d4e3f9a9b54e3c79b597b" member_type: VOTER last_known_addr { host: "127.24.227.252" port: 43385 } }
I20260923 23:18:39.857152 15644 sys_catalog.cc:564] T 00000000000000000000000000000000 P e8135a3a36784cb6b7337e5488aca184 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 23:18:39.857180 15618 raft_consensus.cc:482] T 00000000000000000000000000000000 P e8135a3a36784cb6b7337e5488aca184 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260923 23:18:39.857242 15618 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e8135a3a36784cb6b7337e5488aca184 [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:18:39.857929 15618 raft_consensus.cc:504] T 00000000000000000000000000000000 P e8135a3a36784cb6b7337e5488aca184 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8135a3a36784cb6b7337e5488aca184" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41375 } } peers { permanent_uuid: "8caa054b89104235a283f84b8ef3f691" member_type: VOTER last_known_addr { host: "127.24.227.253" port: 35927 } } peers { permanent_uuid: "73a1f9bc7f5d4e3f9a9b54e3c79b597b" member_type: VOTER last_known_addr { host: "127.24.227.252" port: 43385 } }
I20260923 23:18:39.858130 15618 leader_election.cc:288] T 00000000000000000000000000000000 P e8135a3a36784cb6b7337e5488aca184 [CANDIDATE]: Term 1 election: Requested vote from peers 8caa054b89104235a283f84b8ef3f691 (127.24.227.253:35927), 73a1f9bc7f5d4e3f9a9b54e3c79b597b (127.24.227.252:43385)
I20260923 23:18:39.858364 15768 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e8135a3a36784cb6b7337e5488aca184" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "73a1f9bc7f5d4e3f9a9b54e3c79b597b"
I20260923 23:18:39.858477 15768 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 73a1f9bc7f5d4e3f9a9b54e3c79b597b [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:18:39.859354 15768 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 73a1f9bc7f5d4e3f9a9b54e3c79b597b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8135a3a36784cb6b7337e5488aca184 in term 1.
I20260923 23:18:39.859620 15593 leader_election.cc:302] T 00000000000000000000000000000000 P e8135a3a36784cb6b7337e5488aca184 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73a1f9bc7f5d4e3f9a9b54e3c79b597b, e8135a3a36784cb6b7337e5488aca184; no voters:
I20260923 23:18:39.860005 15831 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e8135a3a36784cb6b7337e5488aca184 [term 1 FOLLOWER]: Leader election won for term 1
W20260923 23:18:39.858871 15825 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e8135a3a36784cb6b7337e5488aca184: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260923 23:18:39.860157 15831 raft_consensus.cc:686] T 00000000000000000000000000000000 P e8135a3a36784cb6b7337e5488aca184 [term 1 LEADER]: Becoming Leader. State: Replica: e8135a3a36784cb6b7337e5488aca184, State: Running, Role: LEADER
W20260923 23:18:39.860318 15825 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 23:18:39.860407 15831 consensus_queue.cc:237] T 00000000000000000000000000000000 P e8135a3a36784cb6b7337e5488aca184 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8135a3a36784cb6b7337e5488aca184" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41375 } } peers { permanent_uuid: "8caa054b89104235a283f84b8ef3f691" member_type: VOTER last_known_addr { host: "127.24.227.253" port: 35927 } } peers { permanent_uuid: "73a1f9bc7f5d4e3f9a9b54e3c79b597b" member_type: VOTER last_known_addr { host: "127.24.227.252" port: 43385 } }
I20260923 23:18:39.860569 15814 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8135a3a36784cb6b7337e5488aca184 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8135a3a36784cb6b7337e5488aca184" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41375 } } peers { permanent_uuid: "8caa054b89104235a283f84b8ef3f691" member_type: VOTER last_known_addr { host: "127.24.227.253" port: 35927 } } peers { permanent_uuid: "73a1f9bc7f5d4e3f9a9b54e3c79b597b" member_type: VOTER last_known_addr { host: "127.24.227.252" port: 43385 } } }
I20260923 23:18:39.860762 15814 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8135a3a36784cb6b7337e5488aca184 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 23:18:39.860850 15831 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8135a3a36784cb6b7337e5488aca184 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e8135a3a36784cb6b7337e5488aca184. Latest consensus state: current_term: 1 leader_uuid: "e8135a3a36784cb6b7337e5488aca184" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8135a3a36784cb6b7337e5488aca184" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41375 } } peers { permanent_uuid: "8caa054b89104235a283f84b8ef3f691" member_type: VOTER last_known_addr { host: "127.24.227.253" port: 35927 } } peers { permanent_uuid: "73a1f9bc7f5d4e3f9a9b54e3c79b597b" member_type: VOTER last_known_addr { host: "127.24.227.252" port: 43385 } } }
I20260923 23:18:39.861857 15831 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8135a3a36784cb6b7337e5488aca184 [sys.catalog]: This master's current role is: LEADER
I20260923 23:18:39.862278 15834 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 23:18:39.862561 15834 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 23:18:39.862112 15576 master_runner.cc:186] Error getting master registration for 127.24.227.253:35927: OK, instance_id { permanent_uuid: "8caa054b89104235a283f84b8ef3f691" instance_seqno: 1790205519650875 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260923 23:18:39.864431 15576 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260923 23:18:39.876277 15647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:39.864115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58906 (local address 127.24.227.254:41375)
0923 23:18:39.864149 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:39.864151 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:39.864472 (+ 321us) server_negotiation.cc:407] Connection header received
0923 23:18:39.864697 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:39.864698 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:39.864707 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:39.864752 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:39.865862 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:39.866567 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:39.868003 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:39.868288 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:39.868323 (+ 35us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:39.872031 (+ 3708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:18:39.872032 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:18:39.872033 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:18:39.872039 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:18:39.873657 (+ 1618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:39.874339 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:39.874340 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:39.874341 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:39.874407 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:39.874893 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:39.874894 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:39.874894 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:39.875014 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:18:39.875089 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:39.876126 (+ 1037us) server_negotiation.cc:299] Negotiation successful
0923 23:18:39.876197 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260923 23:18:39.876355 15722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:39.864227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56088 (local address 127.24.227.253:35927)
0923 23:18:39.864261 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:39.864263 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:39.868041 (+ 3778us) server_negotiation.cc:407] Connection header received
0923 23:18:39.868953 (+ 912us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:39.868955 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:39.868965 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:39.868993 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:39.869759 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:39.870482 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:39.871455 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:39.871733 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:39.871771 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:39.873641 (+ 1870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:18:39.873642 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:18:39.873642 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:18:39.873649 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:18:39.875182 (+ 1533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:39.875566 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:39.875567 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:39.875567 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:39.875614 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:39.875796 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:39.875797 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:39.875797 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:39.875934 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:18:39.875997 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:39.876198 (+ 201us) server_negotiation.cc:299] Negotiation successful
0923 23:18:39.876270 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260923 23:18:39.878132 15768 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 73a1f9bc7f5d4e3f9a9b54e3c79b597b [term 1 FOLLOWER]: Refusing update from remote peer e8135a3a36784cb6b7337e5488aca184: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 23:18:39.878566 15832 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e8135a3a36784cb6b7337e5488aca184 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73a1f9bc7f5d4e3f9a9b54e3c79b597b" member_type: VOTER last_known_addr { host: "127.24.227.252" port: 43385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 23:18:39.879838 15800 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73a1f9bc7f5d4e3f9a9b54e3c79b597b [sys.catalog]: SysCatalogTable state changed. Reason: New leader e8135a3a36784cb6b7337e5488aca184. Latest consensus state: current_term: 1 leader_uuid: "e8135a3a36784cb6b7337e5488aca184" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8135a3a36784cb6b7337e5488aca184" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41375 } } peers { permanent_uuid: "8caa054b89104235a283f84b8ef3f691" member_type: VOTER last_known_addr { host: "127.24.227.253" port: 35927 } } peers { permanent_uuid: "73a1f9bc7f5d4e3f9a9b54e3c79b597b" member_type: VOTER last_known_addr { host: "127.24.227.252" port: 43385 } } }
I20260923 23:18:39.879951 15800 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73a1f9bc7f5d4e3f9a9b54e3c79b597b [sys.catalog]: This master's current role is: FOLLOWER
I20260923 23:18:39.881541 15719 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.227.252" port: 43385 } attempt: 3
I20260923 23:18:39.882308 15797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:39.864436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49630 (local address 127.24.227.252:43385)
0923 23:18:39.864467 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:39.864469 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:39.868136 (+ 3667us) server_negotiation.cc:407] Connection header received
0923 23:18:39.868230 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:39.868231 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:39.868239 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:39.868265 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:39.870072 (+ 1807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:39.870773 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:39.875591 (+ 4818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:39.875917 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:39.875954 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:39.879012 (+ 3058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:18:39.879013 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:18:39.879014 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:18:39.879031 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:18:39.880683 (+ 1652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:39.881234 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:39.881235 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:39.881235 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:39.881297 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:39.881567 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:39.881568 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:39.881568 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:39.881690 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:18:39.881764 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:39.882155 (+ 391us) server_negotiation.cc:299] Negotiation successful
0923 23:18:39.882225 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
Sep 23 23:18:39 dist-test-slave-754n krb5kdc[15563](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790205519, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.227.253@KRBTEST.COM for kudu/127.24.227.252@KRBTEST.COM
I20260923 23:18:39.889731 15814 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8135a3a36784cb6b7337e5488aca184 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e8135a3a36784cb6b7337e5488aca184" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8135a3a36784cb6b7337e5488aca184" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41375 } } peers { permanent_uuid: "8caa054b89104235a283f84b8ef3f691" member_type: VOTER last_known_addr { host: "127.24.227.253" port: 35927 } } peers { permanent_uuid: "73a1f9bc7f5d4e3f9a9b54e3c79b597b" member_type: VOTER last_known_addr { host: "127.24.227.252" port: 43385 } } }
I20260923 23:18:39.889815 15814 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8135a3a36784cb6b7337e5488aca184 [sys.catalog]: This master's current role is: LEADER
I20260923 23:18:39.889947 15838 mvcc.cc:205] Tried to move back new op lower bound from 7332681809416617984 to 7332681809349074944. Current Snapshot: MvccSnapshot[applied={T|T < 7332681809416617984}]
I20260923 23:18:39.889935 15800 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73a1f9bc7f5d4e3f9a9b54e3c79b597b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e8135a3a36784cb6b7337e5488aca184" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8135a3a36784cb6b7337e5488aca184" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41375 } } peers { permanent_uuid: "8caa054b89104235a283f84b8ef3f691" member_type: VOTER last_known_addr { host: "127.24.227.253" port: 35927 } } peers { permanent_uuid: "73a1f9bc7f5d4e3f9a9b54e3c79b597b" member_type: VOTER last_known_addr { host: "127.24.227.252" port: 43385 } } }
I20260923 23:18:39.890005 15800 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73a1f9bc7f5d4e3f9a9b54e3c79b597b [sys.catalog]: This master's current role is: FOLLOWER
I20260923 23:18:39.890055 15839 mvcc.cc:205] Tried to move back new op lower bound from 7332681809416617984 to 7332681809349074944. Current Snapshot: MvccSnapshot[applied={T|T < 7332681809416617984}]
I20260923 23:18:39.890201 15721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:39.881738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.227.252:43385 (local address 127.0.0.1:49634)
0923 23:18:39.881806 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0923 23:18:39.881815 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 23:18:39.882003 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 23:18:39.882478 (+ 475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 23:18:39.882480 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 23:18:39.882628 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0923 23:18:39.882663 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:39.882665 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:39.883552 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:39.883553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:39.884389 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 23:18:39.884392 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 23:18:39.884793 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 23:18:39.884793 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 23:18:39.884859 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:39.885211 (+ 352us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 23:18:39.885215 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 23:18:39.886604 (+ 1389us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 23:18:39.888238 (+ 1634us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:18:39.888239 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:18:39.888239 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:18:39.888446 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:18:39.889074 (+ 628us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 23:18:39.889075 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 23:18:39.889075 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 23:18:39.889170 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 23:18:39.889602 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 23:18:39.889603 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 23:18:39.889661 (+ 58us) client_negotiation.cc:769] Sending connection context
0923 23:18:39.889732 (+ 71us) client_negotiation.cc:240] Negotiation successful
0923 23:18:39.889911 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260923 23:18:39.890754 15834 catalog_manager.cc:1347] Generated new cluster ID: 1cb555b7c6c74dfa9a2df9b37aefd32b
I20260923 23:18:39.890801 15834 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 23:18:39.890961 15797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:39.881911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49634 (local address 127.24.227.252:43385)
0923 23:18:39.882371 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:39.882374 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:39.882381 (+ 7us) server_negotiation.cc:407] Connection header received
0923 23:18:39.882402 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:39.882403 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:39.882412 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:39.882473 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:39.882761 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:39.883475 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:39.884461 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:39.884730 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:39.884792 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:39.886707 (+ 1915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 23:18:39.886708 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 23:18:39.886709 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 23:18:39.886718 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 23:18:39.888160 (+ 1442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:39.888886 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:39.888887 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:39.888887 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:39.888962 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 23:18:39.889256 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 23:18:39.889257 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 23:18:39.889258 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 23:18:39.889454 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 23:18:39.890563 (+ 1109us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:39.890626 (+ 63us) server_negotiation.cc:299] Negotiation successful
0923 23:18:39.890745 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":446,"thread_start_us":67,"threads_started":1}
I20260923 23:18:39.892496 15719 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8caa054b89104235a283f84b8ef3f691: Bootstrap starting.
I20260923 23:18:39.893138 15719 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8caa054b89104235a283f84b8ef3f691: Neither blocks nor log segments found. Creating new log.
I20260923 23:18:39.893412 15719 log.cc:824] T 00000000000000000000000000000000 P 8caa054b89104235a283f84b8ef3f691: Log is configured to *not* fsync() on all Append() calls
I20260923 23:18:39.904780 15719 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8caa054b89104235a283f84b8ef3f691: No bootstrap required, opened a new log
I20260923 23:18:39.906607 15719 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8caa054b89104235a283f84b8ef3f691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8135a3a36784cb6b7337e5488aca184" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41375 } } peers { permanent_uuid: "8caa054b89104235a283f84b8ef3f691" member_type: VOTER last_known_addr { host: "127.24.227.253" port: 35927 } } peers { permanent_uuid: "73a1f9bc7f5d4e3f9a9b54e3c79b597b" member_type: VOTER last_known_addr { host: "127.24.227.252" port: 43385 } }
I20260923 23:18:39.906729 15719 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8caa054b89104235a283f84b8ef3f691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8caa054b89104235a283f84b8ef3f691, State: Initialized, Role: FOLLOWER
I20260923 23:18:39.906800 15719 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8caa054b89104235a283f84b8ef3f691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8135a3a36784cb6b7337e5488aca184" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41375 } } peers { permanent_uuid: "8caa054b89104235a283f84b8ef3f691" member_type: VOTER last_known_addr { host: "127.24.227.253" port: 35927 } } peers { permanent_uuid: "73a1f9bc7f5d4e3f9a9b54e3c79b597b" member_type: VOTER last_known_addr { host: "127.24.227.252" port: 43385 } }
I20260923 23:18:39.907080 15719 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8caa054b89104235a283f84b8ef3f691 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 23:18:39.907179 15692 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8caa054b89104235a283f84b8ef3f691 [term 0 FOLLOWER]: Advancing to term 1
I20260923 23:18:39.908002 15692 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8caa054b89104235a283f84b8ef3f691 [term 1 FOLLOWER]: Refusing update from remote peer e8135a3a36784cb6b7337e5488aca184: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 23:18:39.908183 15693 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e8135a3a36784cb6b7337e5488aca184" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "8caa054b89104235a283f84b8ef3f691"
I20260923 23:18:39.908566 15834 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 23:18:39.908995 15693 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8caa054b89104235a283f84b8ef3f691 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8135a3a36784cb6b7337e5488aca184 in term 1.
I20260923 23:18:39.909047 15814 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e8135a3a36784cb6b7337e5488aca184 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8caa054b89104235a283f84b8ef3f691" member_type: VOTER last_known_addr { host: "127.24.227.253" port: 35927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 23:18:39.907078 15845 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8caa054b89104235a283f84b8ef3f691 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8135a3a36784cb6b7337e5488aca184" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41375 } } peers { permanent_uuid: "8caa054b89104235a283f84b8ef3f691" member_type: VOTER last_known_addr { host: "127.24.227.253" port: 35927 } } peers { permanent_uuid: "73a1f9bc7f5d4e3f9a9b54e3c79b597b" member_type: VOTER last_known_addr { host: "127.24.227.252" port: 43385 } } }
I20260923 23:18:39.909366 15845 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8caa054b89104235a283f84b8ef3f691 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 23:18:39.910590 15856 mvcc.cc:205] Tried to move back new op lower bound from 7332681809416617984 to 7332681809349074944. Current Snapshot: MvccSnapshot[applied={T|T < 7332681809416617984}]
I20260923 23:18:39.911098 15834 catalog_manager.cc:1504] Loading token signing keys...
I20260923 23:18:39.911096 15845 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8caa054b89104235a283f84b8ef3f691 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e8135a3a36784cb6b7337e5488aca184. Latest consensus state: current_term: 1 leader_uuid: "e8135a3a36784cb6b7337e5488aca184" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8135a3a36784cb6b7337e5488aca184" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41375 } } peers { permanent_uuid: "8caa054b89104235a283f84b8ef3f691" member_type: VOTER last_known_addr { host: "127.24.227.253" port: 35927 } } peers { permanent_uuid: "73a1f9bc7f5d4e3f9a9b54e3c79b597b" member_type: VOTER last_known_addr { host: "127.24.227.252" port: 43385 } } }
I20260923 23:18:39.911257 15845 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8caa054b89104235a283f84b8ef3f691 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 23:18:39.911325 15845 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8caa054b89104235a283f84b8ef3f691 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e8135a3a36784cb6b7337e5488aca184" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8135a3a36784cb6b7337e5488aca184" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41375 } } peers { permanent_uuid: "8caa054b89104235a283f84b8ef3f691" member_type: VOTER last_known_addr { host: "127.24.227.253" port: 35927 } } peers { permanent_uuid: "73a1f9bc7f5d4e3f9a9b54e3c79b597b" member_type: VOTER last_known_addr { host: "127.24.227.252" port: 43385 } } }
I20260923 23:18:39.912216 15845 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8caa054b89104235a283f84b8ef3f691 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 23:18:39.916532 15814 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8135a3a36784cb6b7337e5488aca184 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e8135a3a36784cb6b7337e5488aca184" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8135a3a36784cb6b7337e5488aca184" member_type: VOTER last_known_addr { host: "127.24.227.254" port: 41375 } } peers { permanent_uuid: "8caa054b89104235a283f84b8ef3f691" member_type: VOTER last_known_addr { host: "127.24.227.253" port: 35927 } } peers { permanent_uuid: "73a1f9bc7f5d4e3f9a9b54e3c79b597b" member_type: VOTER last_known_addr { host: "127.24.227.252" port: 43385 } } }
I20260923 23:18:39.916625 15814 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8135a3a36784cb6b7337e5488aca184 [sys.catalog]: This master's current role is: LEADER
I20260923 23:18:39.916829 15861 catalog_manager.cc:1259] Loaded cluster ID: 1cb555b7c6c74dfa9a2df9b37aefd32b
I20260923 23:18:39.916867 15861 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8caa054b89104235a283f84b8ef3f691: loading cluster ID for follower catalog manager: success
I20260923 23:18:39.917873 15861 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8caa054b89104235a283f84b8ef3f691: acquiring CA information for follower catalog manager: success
I20260923 23:18:39.918207 15861 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8caa054b89104235a283f84b8ef3f691: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260923 23:18:39.919972 15834 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e8135a3a36784cb6b7337e5488aca184: Generated new TSK 0
I20260923 23:18:39.920110 15834 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260923 23:18:39.944614 15722 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 23:18:39.943075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56092 (local address 127.24.227.253:35927)
0923 23:18:39.943115 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:39.943117 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:39.944009 (+ 892us) server_negotiation.cc:407] Connection header received
0923 23:18:39.944100 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:39.944101 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:39.944111 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:39.944140 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:39.944559 (+ 419us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56092: BlockingRecv error: recv got EOF from 127.0.0.1:56092 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260923 23:18:39.945313 15647 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 23:18:39.943448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58914 (local address 127.24.227.254:41375)
0923 23:18:39.943483 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:39.943485 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:39.944845 (+ 1360us) server_negotiation.cc:407] Connection header received
0923 23:18:39.944943 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:39.944944 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:39.944953 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:39.944980 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:39.945263 (+ 283us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58914: BlockingRecv error: recv got EOF from 127.0.0.1:58914 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260923 23:18:39.946045 15797 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 23:18:39.943698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49650 (local address 127.24.227.252:43385)
0923 23:18:39.943733 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:39.943735 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:39.945498 (+ 1763us) server_negotiation.cc:407] Connection header received
0923 23:18:39.945647 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:39.945648 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:39.945657 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:39.945686 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:39.945993 (+ 307us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49650: BlockingRecv error: recv got EOF from 127.0.0.1:49650 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
I20260923 23:18:39.950812 15722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:39.947513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56108 (local address 127.24.227.253:35927)
0923 23:18:39.947545 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:39.947546 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:39.947717 (+ 171us) server_negotiation.cc:407] Connection header received
0923 23:18:39.947810 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:39.947811 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:39.947842 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:39.947908 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 23:18:39.948085 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:39.948770 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:39.949895 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:39.950162 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:39.950220 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:39.950449 (+ 229us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 23:18:39.950551 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 23:18:39.950598 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:39.950740 (+ 142us) server_negotiation.cc:299] Negotiation successful
0923 23:18:39.950757 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
W20260923 23:18:39.953647 15797 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 23:18:39.952846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49662 (local address 127.24.227.252:43385)
0923 23:18:39.952878 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:39.952879 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:39.953134 (+ 255us) server_negotiation.cc:407] Connection header received
0923 23:18:39.953253 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:39.953254 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:39.953269 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:39.953300 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:39.953605 (+ 305us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49662: BlockingRecv error: recv got EOF from 127.0.0.1:49662 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
I20260923 23:18:39.958776 15722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:39.955748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56114 (local address 127.24.227.253:35927)
0923 23:18:39.955786 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:39.955788 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:39.956015 (+ 227us) server_negotiation.cc:407] Connection header received
0923 23:18:39.956091 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:39.956092 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:39.956099 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:39.956121 (+ 22us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 23:18:39.956267 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:39.956943 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:39.957946 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:39.958249 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:39.958309 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:39.958549 (+ 240us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 23:18:39.958578 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 23:18:39.958604 (+ 26us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:39.958702 (+ 98us) server_negotiation.cc:299] Negotiation successful
0923 23:18:39.958718 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
W20260923 23:18:39.961634 15797 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 23:18:39.960909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49664 (local address 127.24.227.252:43385)
0923 23:18:39.960940 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:39.960941 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:39.961149 (+ 208us) server_negotiation.cc:407] Connection header received
0923 23:18:39.961259 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:39.961260 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:39.961269 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:39.961299 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:39.961595 (+ 296us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49664: BlockingRecv error: recv got EOF from 127.0.0.1:49664 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
I20260923 23:18:39.967625 15722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:39.964629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56118 (local address 127.24.227.253:35927)
0923 23:18:39.964672 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:39.964674 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:39.964909 (+ 235us) server_negotiation.cc:407] Connection header received
0923 23:18:39.964997 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:39.964998 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:39.965006 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:39.965032 (+ 26us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 23:18:39.965198 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:39.965842 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:39.966853 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:39.967129 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:39.967173 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:39.967389 (+ 216us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 23:18:39.967418 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 23:18:39.967445 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:39.967550 (+ 105us) server_negotiation.cc:299] Negotiation successful
0923 23:18:39.967566 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
W20260923 23:18:39.970463 15797 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 23:18:39.969735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49672 (local address 127.24.227.252:43385)
0923 23:18:39.969769 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:39.969771 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:39.969955 (+ 184us) server_negotiation.cc:407] Connection header received
0923 23:18:39.970050 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:39.970051 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:39.970061 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:39.970090 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:39.970420 (+ 330us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49672: BlockingRecv error: recv got EOF from 127.0.0.1:49672 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
I20260923 23:18:39.978545 15722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:39.975472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56120 (local address 127.24.227.253:35927)
0923 23:18:39.975504 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:39.975506 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:39.975731 (+ 225us) server_negotiation.cc:407] Connection header received
0923 23:18:39.975822 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:39.975823 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:39.975831 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:39.975857 (+ 26us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 23:18:39.976063 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:39.976714 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:39.977731 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:39.978018 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:39.978059 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:39.978306 (+ 247us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 23:18:39.978336 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 23:18:39.978364 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:39.978468 (+ 104us) server_negotiation.cc:299] Negotiation successful
0923 23:18:39.978484 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
W20260923 23:18:39.981563 15797 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 23:18:39.980868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49680 (local address 127.24.227.252:43385)
0923 23:18:39.980898 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:39.980900 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:39.981078 (+ 178us) server_negotiation.cc:407] Connection header received
0923 23:18:39.981168 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:39.981169 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:39.981178 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:39.981220 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:39.981522 (+ 302us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49680: BlockingRecv error: recv got EOF from 127.0.0.1:49680 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
I20260923 23:18:39.993829 15722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:39.990671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56130 (local address 127.24.227.253:35927)
0923 23:18:39.990703 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:39.990705 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:39.990922 (+ 217us) server_negotiation.cc:407] Connection header received
0923 23:18:39.991007 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:39.991008 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:39.991016 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:39.991041 (+ 25us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 23:18:39.991261 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:39.992023 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:39.993081 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:39.993372 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:39.993411 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:39.993592 (+ 181us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 23:18:39.993620 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 23:18:39.993645 (+ 25us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:39.993748 (+ 103us) server_negotiation.cc:299] Negotiation successful
0923 23:18:39.993765 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
W20260923 23:18:39.996650 15797 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 23:18:39.995935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49690 (local address 127.24.227.252:43385)
0923 23:18:39.995972 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:39.995973 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:39.996152 (+ 179us) server_negotiation.cc:407] Connection header received
0923 23:18:39.996256 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:39.996257 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:39.996266 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:39.996294 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:39.996604 (+ 310us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49690: BlockingRecv error: recv got EOF from 127.0.0.1:49690 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
I20260923 23:18:40.016929 15722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:40.013879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56132 (local address 127.24.227.253:35927)
0923 23:18:40.013920 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:40.013922 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:40.014089 (+ 167us) server_negotiation.cc:407] Connection header received
0923 23:18:40.014124 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:40.014125 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:40.014136 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:40.014201 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 23:18:40.014373 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:40.015104 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:40.016192 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:40.016465 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:40.016538 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:40.016685 (+ 147us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 23:18:40.016717 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 23:18:40.016771 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:40.016844 (+ 73us) server_negotiation.cc:299] Negotiation successful
0923 23:18:40.016860 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
W20260923 23:18:40.019408 15797 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 23:18:40.018825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49694 (local address 127.24.227.252:43385)
0923 23:18:40.018863 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:40.018864 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:40.018875 (+ 11us) server_negotiation.cc:407] Connection header received
0923 23:18:40.018957 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:40.018958 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:40.018967 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:40.019033 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:40.019361 (+ 328us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49694: BlockingRecv error: recv got EOF from 127.0.0.1:49694 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 23:18:40.055125 15722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:40.052490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56148 (local address 127.24.227.253:35927)
0923 23:18:40.052554 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:40.052556 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:40.052682 (+ 126us) server_negotiation.cc:407] Connection header received
0923 23:18:40.052713 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:40.052714 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:40.052726 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:40.052785 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 23:18:40.052909 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:40.053655 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:40.054389 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:40.054688 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:40.054762 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:40.054882 (+ 120us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 23:18:40.054914 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 23:18:40.054967 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:40.055025 (+ 58us) server_negotiation.cc:299] Negotiation successful
0923 23:18:40.055046 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
W20260923 23:18:40.057541 15797 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 23:18:40.056868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49698 (local address 127.24.227.252:43385)
0923 23:18:40.056923 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:40.056925 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:40.056980 (+ 55us) server_negotiation.cc:407] Connection header received
0923 23:18:40.057057 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:40.057057 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:40.057069 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:40.057125 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:40.057472 (+ 347us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49698: BlockingRecv error: recv got EOF from 127.0.0.1:49698 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
I20260923 23:18:40.124837 15722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:40.122676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56156 (local address 127.24.227.253:35927)
0923 23:18:40.122754 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:40.122756 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:40.122832 (+ 76us) server_negotiation.cc:407] Connection header received
0923 23:18:40.122876 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:40.122877 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:40.122886 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:40.122936 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 23:18:40.123072 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:40.123542 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:40.124299 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:40.124458 (+ 159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:40.124503 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:40.124605 (+ 102us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 23:18:40.124628 (+ 23us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 23:18:40.124661 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:40.124707 (+ 46us) server_negotiation.cc:299] Negotiation successful
0923 23:18:40.124720 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
W20260923 23:18:40.126989 15797 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 23:18:40.126471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49702 (local address 127.24.227.252:43385)
0923 23:18:40.126546 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:40.126548 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:40.126580 (+ 32us) server_negotiation.cc:407] Connection header received
0923 23:18:40.126627 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:40.126627 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:40.126636 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:40.126680 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:40.126943 (+ 263us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49702: BlockingRecv error: recv got EOF from 127.0.0.1:49702 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
I20260923 23:18:40.258756 15722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:40.256225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56160 (local address 127.24.227.253:35927)
0923 23:18:40.256309 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:40.256311 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:40.256406 (+ 95us) server_negotiation.cc:407] Connection header received
0923 23:18:40.256503 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:40.256504 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:40.256514 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:40.256590 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 23:18:40.256725 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:40.257282 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:40.258110 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:40.258309 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:40.258376 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:40.258509 (+ 133us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 23:18:40.258543 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 23:18:40.258604 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:40.258653 (+ 49us) server_negotiation.cc:299] Negotiation successful
0923 23:18:40.258671 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
W20260923 23:18:40.260860 15797 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 23:18:40.260309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49714 (local address 127.24.227.252:43385)
0923 23:18:40.260363 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:40.260365 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:40.260404 (+ 39us) server_negotiation.cc:407] Connection header received
0923 23:18:40.260473 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:40.260474 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:40.260483 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:40.260533 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:40.260804 (+ 271us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49714: BlockingRecv error: recv got EOF from 127.0.0.1:49714 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
I20260923 23:18:40.519809 15722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:40.517858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56170 (local address 127.24.227.253:35927)
0923 23:18:40.517930 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:40.517932 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:40.517973 (+ 41us) server_negotiation.cc:407] Connection header received
0923 23:18:40.518012 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:40.518013 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:40.518022 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:40.518076 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 23:18:40.518203 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:40.518624 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:40.519302 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:40.519468 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:40.519515 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:40.519634 (+ 119us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 23:18:40.519658 (+ 24us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 23:18:40.519700 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:40.519742 (+ 42us) server_negotiation.cc:299] Negotiation successful
0923 23:18:40.519756 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
W20260923 23:18:40.521896 15797 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 23:18:40.521404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49730 (local address 127.24.227.252:43385)
0923 23:18:40.521462 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:40.521464 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:40.521473 (+ 9us) server_negotiation.cc:407] Connection header received
0923 23:18:40.521500 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:40.521501 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:40.521511 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:40.521555 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0923 23:18:40.521843 (+ 288us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49730: BlockingRecv error: recv got EOF from 127.0.0.1:49730 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
I20260923 23:18:40.830981 15811 catalog_manager.cc:1259] Loaded cluster ID: 1cb555b7c6c74dfa9a2df9b37aefd32b
I20260923 23:18:40.831075 15811 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 73a1f9bc7f5d4e3f9a9b54e3c79b597b: loading cluster ID for follower catalog manager: success
I20260923 23:18:40.831832 15811 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 73a1f9bc7f5d4e3f9a9b54e3c79b597b: acquiring CA information for follower catalog manager: success
I20260923 23:18:40.832084 15811 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 73a1f9bc7f5d4e3f9a9b54e3c79b597b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260923 23:18:41.037467 15977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:41.034916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56172 (local address 127.24.227.253:35927)
0923 23:18:41.035042 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:41.035044 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:41.035130 (+ 86us) server_negotiation.cc:407] Connection header received
0923 23:18:41.035246 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:41.035247 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:41.035260 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:41.035340 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 23:18:41.035474 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:41.035990 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:41.036766 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:41.036968 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:41.037037 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:41.037188 (+ 151us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 23:18:41.037227 (+ 39us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 23:18:41.037282 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:41.037342 (+ 60us) server_negotiation.cc:299] Negotiation successful
0923 23:18:41.037367 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":61,"threads_started":1}
I20260923 23:18:41.041512 15983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:41.039087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49740 (local address 127.24.227.252:43385)
0923 23:18:41.039187 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:41.039189 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:41.039198 (+ 9us) server_negotiation.cc:407] Connection header received
0923 23:18:41.039263 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:41.039264 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:41.039334 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:41.039416 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 23:18:41.039505 (+ 89us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:41.040016 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:41.040733 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:41.040934 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:41.041008 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:41.041170 (+ 162us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 23:18:41.041283 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 23:18:41.041382 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:41.041403 (+ 21us) server_negotiation.cc:299] Negotiation successful
0923 23:18:41.041422 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76,"thread_start_us":42,"threads_started":1}
I20260923 23:18:41.045701 15977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:41.042955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56176 (local address 127.24.227.253:35927)
0923 23:18:41.043016 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:41.043018 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:41.043225 (+ 207us) server_negotiation.cc:407] Connection header received
0923 23:18:41.043289 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:41.043290 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:41.043301 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:41.043349 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 23:18:41.043505 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:41.044091 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:41.045032 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:41.045297 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:41.045354 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:41.045473 (+ 119us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 23:18:41.045498 (+ 25us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 23:18:41.045544 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:41.045604 (+ 60us) server_negotiation.cc:299] Negotiation successful
0923 23:18:41.045622 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 23:18:41.045975 15983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:41.043143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49750 (local address 127.24.227.252:43385)
0923 23:18:41.043205 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:41.043207 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:41.043303 (+ 96us) server_negotiation.cc:407] Connection header received
0923 23:18:41.043403 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:41.043403 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:41.043413 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:41.043478 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 23:18:41.043610 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:41.044343 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:41.045273 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:41.045526 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:41.045577 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:41.045734 (+ 157us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 23:18:41.045761 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 23:18:41.045788 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:41.045897 (+ 109us) server_negotiation.cc:299] Negotiation successful
0923 23:18:41.045917 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260923 23:18:41.046172 15990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 23:18:41.042974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58916 (local address 127.24.227.254:41375)
0923 23:18:41.043338 (+ 364us) server_negotiation.cc:206] Beginning negotiation
0923 23:18:41.043340 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 23:18:41.043348 (+ 8us) server_negotiation.cc:407] Connection header received
0923 23:18:41.043392 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 23:18:41.043393 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 23:18:41.043443 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 23:18:41.043520 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 23:18:41.043665 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:41.044433 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:41.045159 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 23:18:41.045443 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 23:18:41.045496 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 23:18:41.045713 (+ 217us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 23:18:41.045827 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 23:18:41.045876 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0923 23:18:41.046016 (+ 140us) server_negotiation.cc:299] Negotiation successful
0923 23:18:41.046039 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":111,"threads_started":1}
I20260923 23:18:41.055299 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 15576
I20260923 23:18:41.060596 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 15651
I20260923 23:18:41.065255 25487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskyHY_dZ/build/release/bin/kudu with pid 15726
2026-09-23T23:18:41Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2795 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5543 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (261129 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST